2024-05-09T08:43:30.764 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T08:43:30.765 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706 --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} --verbose -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/orig.config.yaml 2024-05-09T08:43:30.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:43:30.899 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698706' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGF0Tu/xeTs7l2x4Qq0xXvIlVrG7D3wHuFlpNwwfRA2iHoSmimelUq72doWmhRYzkhnFxjQm+q0E8mkCrDShKU= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6djY6K06KwtlHhYdcna2V352MkN2KWcUsG5mzROcElNCcx7I6VjhjiGEt4UpIny1Acj2Gk28+l26DFBD9XnPM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - thrashosds: null - workunit: clients: all: - fs/misc/trivial_sync.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T08:43:30.899 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T08:43:30.899 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T08:43:30.900 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T08:43:30.901 INFO:teuthology.task.internal:Checking packages... 2024-05-09T08:43:30.924 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T08:43:30.925 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T08:43:30.925 INFO:teuthology.packaging:ref: None 2024-05-09T08:43:30.925 INFO:teuthology.packaging:tag: None 2024-05-09T08:43:30.925 INFO:teuthology.packaging:branch: squid 2024-05-09T08:43:30.925 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:43:30.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T08:43:31.144 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T08:43:31.145 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T08:43:31.152 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T08:43:31.152 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T08:43:31.157 INFO:teuthology.task.internal:Saving configuration 2024-05-09T08:43:31.176 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T08:43:31.182 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T08:43:31.203 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:37:40.481123', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGF0Tu/xeTs7l2x4Qq0xXvIlVrG7D3wHuFlpNwwfRA2iHoSmimelUq72doWmhRYzkhnFxjQm+q0E8mkCrDShKU='} 2024-05-09T08:43:31.219 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 08:37:40.479575', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6djY6K06KwtlHhYdcna2V352MkN2KWcUsG5mzROcElNCcx7I6VjhjiGEt4UpIny1Acj2Gk28+l26DFBD9XnPM='} 2024-05-09T08:43:31.219 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T08:43:31.225 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-05-09T08:43:31.225 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-05-09T08:43:31.225 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T08:43:31.298 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb69d9a50d0>, signals=[15]) 2024-05-09T08:43:31.298 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T08:43:31.304 INFO:teuthology.task.internal:Opening connections... 2024-05-09T08:43:31.304 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:43:31.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:43:31.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:43:31.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:43:31.453 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T08:43:31.460 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-05-09T08:43:31.479 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-05-09T08:43:31.479 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="CentOS Stream" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="9" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:ID="centos" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="9" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:43:31.537 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:43:31.538 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:43:31.539 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-05-09T08:43:31.560 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-05-09T08:43:31.578 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-05-09T08:43:31.578 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T08:43:31.635 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T08:43:31.636 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T08:43:31.636 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-05-09T08:43:31.660 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T08:43:31.669 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T08:43:31.673 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T08:43:31.673 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:43:31.676 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-05-09T08:43:31.692 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T08:43:31.697 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T08:43:31.697 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:43:31.734 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-05-09T08:43:31.838 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T08:43:31.890 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T08:43:31.927 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T08:43:31.927 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-09T08:43:31.928 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-09T08:43:31.928 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:43:31.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:43:31.946 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-05-09T08:43:32.003 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-437.el9.x86_64 2024-05-09T08:43:32.003 INFO:teuthology.task.kernel:Running kernel on smithi049: 5.14.0-437.el9.x86_64 2024-05-09T08:43:32.004 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-05-09T08:43:34.182 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-09T08:43:36.058 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-09T08:43:41.916 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-05-09T08:43:43.300 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 131 kB/s | 16 kB 00:00 2024-05-09T08:44:02.354 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-05-09T08:44:07.743 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 9.3 kB/s | 1.7 kB 00:00 2024-05-09T08:44:09.857 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T08:44:09.858 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T08:44:09.943 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-09T08:44:09.957 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-09T08:44:09.957 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-05-09T08:44:09.957 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-09T08:44:09.957 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout:Install 4 Packages 2024-05-09T08:44:09.958 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:44:09.959 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 78 M 2024-05-09T08:44:09.959 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 123 M 2024-05-09T08:44:09.959 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-09T08:44:10.122 INFO:teuthology.orchestra.run.smithi049.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 3.7 MB/s | 484 kB 00:00 2024-05-09T08:44:10.545 INFO:teuthology.orchestra.run.smithi049.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 28 MB/s | 15 MB 00:00 2024-05-09T08:44:11.120 INFO:teuthology.orchestra.run.smithi049.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-05-09T08:44:11.512 INFO:teuthology.orchestra.run.smithi049.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-09T08:44:11.512 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:44:11.512 INFO:teuthology.orchestra.run.smithi049.stdout:Total 50 MB/s | 78 MB 00:01 2024-05-09T08:44:12.221 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-09T08:44:12.616 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-09T08:44:12.617 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-09T08:44:13.561 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-09T08:44:13.561 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-09T08:44:16.116 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-09T08:44:16.745 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:44:16.829 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:44:19.636 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:44:19.760 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:44:24.834 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:44:24.879 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:44:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:45:00.777 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:45:00.817 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:45:01.208 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:45:01.209 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T08:45:01.209 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T08:45:01.209 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:45:02.008 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-09T08:45:02.401 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:45:02.492 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.492 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:45:02.492 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T08:45:02.493 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T08:45:02.493 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T08:45:02.493 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T08:45:02.493 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi049.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T08:45:02.493 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T08:45:02.544 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T08:45:02.544 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T08:45:02.545 DEBUG:teuthology.orchestra.run.smithi049:> sudo rpm -qi grub2-tools 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Name : grub2-tools 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Epoch : 1 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Version : 2.06 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Release : 68.el9 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Architecture: x86_64 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Group : Unspecified 2024-05-09T08:45:02.584 INFO:teuthology.orchestra.run.smithi049.stdout:Size : 8264254 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:License : GPLv3+ 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Packager : builder@centos.org 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Vendor : CentOS 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Summary : Support tools for GRUB. 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:Description : 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T08:45:02.585 INFO:teuthology.orchestra.run.smithi049.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T08:45:02.586 INFO:teuthology.orchestra.run.smithi049.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T08:45:02.586 INFO:teuthology.orchestra.run.smithi049.stdout:hardware devices. 2024-05-09T08:45:02.586 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:45:02.586 INFO:teuthology.orchestra.run.smithi049.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T08:45:02.587 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T08:45:02.587 INFO:teuthology.task.kernel:Updating grub on smithi049 to boot 5.14.0-444.el9.x86_64 2024-05-09T08:45:02.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T08:45:03.232 INFO:teuthology.orchestra.run.smithi049.stderr:Generating grub configuration file ... 2024-05-09T08:45:04.668 INFO:teuthology.orchestra.run.smithi049.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T08:45:04.708 INFO:teuthology.orchestra.run.smithi049.stderr:done 2024-05-09T08:45:04.711 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-05-09T08:45:04.767 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.E0DbKLJEAh 2024-05-09T08:45:04.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E0DbKLJEAh 2024-05-09T08:45:04.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.E0DbKLJEAh 2024-05-09T08:45:04.973 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.E0DbKLJEAh is 6KB 2024-05-09T08:45:04.993 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.E0DbKLJEAh 2024-05-09T08:45:05.008 DEBUG:teuthology.orchestra.run.smithi049:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T08:45:05.075 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T08:45:05.075 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T08:45:05.075 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T08:45:05.075 INFO:teuthology.orchestra.run.smithi049.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T08:45:05.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T08:45:05.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo shutdown -r now 2024-05-09T08:45:35.237 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T08:45:35.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:45:35.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:45:53.566 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-09T08:46:02.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:46:02.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:46:05.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-09T08:46:17.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:46:17.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:46:20.738 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-09T08:46:35.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:46:35.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:46:54.245 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.49 2024-05-09T08:47:12.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:47:12.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:47:12.587 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-05-09T08:47:12.995 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:47:12.996 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T08:47:12.996 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-05-09T08:47:13.012 INFO:teuthology.orchestra.run.smithi049.stdout:5.14.0-444.el9.x86_64 2024-05-09T08:47:13.012 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T08:47:13.012 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T08:47:13.012 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T08:47:14.013 DEBUG:teuthology.parallel:result is None 2024-05-09T08:47:14.014 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T08:47:14.022 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T08:47:14.022 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:47:14.024 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T08:47:14.042 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T08:47:14.050 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T08:47:14.060 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T08:47:14.060 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:47:14.084 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T08:47:14.116 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T08:47:14.123 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T08:47:14.124 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T08:47:14.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:14.167 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:47:14.169 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T08:47:14.232 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:47:14.247 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:47:14.251 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:47:14.266 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T08:47:14.268 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T08:47:14.275 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T08:47:14.275 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:47:14.277 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T08:47:14.333 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T08:47:14.343 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T08:47:14.343 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:47:14.347 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T08:47:14.388 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:47:14.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:47:14.508 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:47:14.595 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:47:14.639 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:47:14.640 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:47:14.710 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:47:14.750 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T08:47:14.818 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:47:14.862 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T08:47:14.900 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:47:14.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T08:47:14.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-05-09T08:47:14.965 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-05-09T08:47:15.014 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:47:15.036 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:47:15.481 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T08:47:15.489 INFO:teuthology.task.internal:Starting timer... 2024-05-09T08:47:15.489 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T08:47:15.503 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T08:47:15.516 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T08:47:15.517 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-05-09T08:47:15.624 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-05-09T08:47:15.626 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-05-09T08:47:15.683 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-05-09T08:47:15.685 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T08:47:15.685 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-05-09T08:47:15.711 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-05-09T08:47:15.711 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-05-09T08:47:15.732 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-05-09T08:47:15.732 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-05-09T08:47:15.732 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:47:15.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:15.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:47:15.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:47:15.813 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T08:47:15.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:47:15.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-05-09T08:47:15.895 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T08:47:15.906 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-09T08:47:15.911 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-09T08:47:16.035 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T08:47:16.051 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T08:47:16.053 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T08:51:32.127 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-05-09T08:51:32.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:51:32.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:51:32.204 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-05-09T08:51:32.283 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-05-09T08:51:32.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-09T08:51:32.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T08:51:32.359 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-05-09T08:51:32.436 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-09T08:51:32.437 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T08:51:32.448 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T08:51:32.449 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:51:32.449 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:51:32.452 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T08:51:32.452 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:51:32.480 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:51:32.495 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:51:32.504 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T08:51:32.517 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T08:51:32.520 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-05-09T08:51:32.531 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-05-09T08:51:32.538 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-05-09T08:51:32.546 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:51:32.549 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-05-09T08:51:32.560 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:51:32.562 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T08:51:32.575 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T08:51:32.628 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.632 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.641 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:51:32.644 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:51:32.645 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-05-09T08:51:32.645 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.645 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.645 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.645 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:51:32.646 INFO:teuthology.run_tasks:Running task install... 2024-05-09T08:51:32.654 DEBUG:teuthology.task.install:project ceph 2024-05-09T08:51:32.654 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T08:51:32.654 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T08:51:32.655 INFO:teuthology.task.install:Using flavor: default 2024-05-09T08:51:32.660 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T08:51:32.660 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T08:51:32.661 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:51:32.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:51:32.663 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T08:51:32.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:51:32.808 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:51:32.808 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:51:32.813 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T08:51:32.813 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T08:51:32.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:51:32.931 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:51:32.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:51:32.958 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:51:32.958 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:51:33.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:51:33.100 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T08:51:33.100 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:51:33.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T08:51:33.102 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:51:33.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T08:51:33.124 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T08:51:33.167 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-05-09T08:51:33.168 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-05-09T08:51:33.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T08:51:33.260 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T08:51:33.323 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-05-09T08:51:33.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-05-09T08:51:33.410 INFO:teuthology.orchestra.run.smithi049.stdout:54 files removed 2024-05-09T08:51:33.437 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:51:33.551 INFO:teuthology.orchestra.run.smithi183.stdout:54 files removed 2024-05-09T08:51:33.579 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T08:51:34.294 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 147 kB/s | 83 kB 00:00 2024-05-09T08:51:34.346 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 165 kB/s | 83 kB 00:00 2024-05-09T08:51:34.803 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-05-09T08:51:34.830 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-05-09T08:51:35.295 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-05-09T08:51:35.297 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-05-09T08:51:37.437 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.1 MB 00:02 2024-05-09T08:51:37.864 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.1 MB 00:02 2024-05-09T08:51:39.380 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-09T08:51:47.948 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:08 2024-05-09T08:51:52.446 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - CRB 679 kB/s | 6.2 MB 00:09 2024-05-09T08:51:54.431 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-09T08:51:55.931 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-05-09T08:51:59.960 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 5.4 MB/s | 21 MB 00:03 2024-05-09T08:52:05.049 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T08:52:07.351 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:52:07.351 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:52:07.362 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:52:07.363 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:52:07.364 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:52:07.365 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:52:07.366 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:52:07.369 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:52:07.369 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:52:07.430 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:52:07.447 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:52:07.448 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:52:07.449 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:52:07.450 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:52:07.451 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:52:07.452 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:52:07.453 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:52:07.454 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:52:07.455 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:52:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:52:07.457 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:52:07.458 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-05-09T08:52:07.459 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:52:07.460 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 225 M 2024-05-09T08:52:07.460 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-09T08:52:08.476 INFO:teuthology.orchestra.run.smithi183.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-09T08:52:10.526 INFO:teuthology.orchestra.run.smithi183.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 434 kB/s | 889 kB 00:02 2024-05-09T08:52:10.609 INFO:teuthology.orchestra.run.smithi183.stdout:(3/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-05-09T08:52:10.826 INFO:teuthology.orchestra.run.smithi183.stdout:(4/156): ceph-immutable-object-cache-19.0.0-251 483 kB/s | 145 kB 00:00 2024-05-09T08:52:10.901 INFO:teuthology.orchestra.run.smithi183.stdout:(5/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-05-09T08:52:11.452 INFO:teuthology.orchestra.run.smithi183.stdout:(6/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 8.5 MB/s | 4.7 MB 00:00 2024-05-09T08:52:13.068 INFO:teuthology.orchestra.run.smithi183.stdout:(7/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 10 MB/s | 17 MB 00:01 2024-05-09T08:52:13.635 INFO:teuthology.orchestra.run.smithi183.stdout:(8/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 545 kB/s | 1.5 MB 00:02 2024-05-09T08:52:13.761 INFO:teuthology.orchestra.run.smithi183.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 201 kB/s | 25 kB 00:00 2024-05-09T08:52:14.011 INFO:teuthology.orchestra.run.smithi183.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 11 MB/s | 10 MB 00:00 2024-05-09T08:52:14.061 INFO:teuthology.orchestra.run.smithi183.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 4.4 MB/s | 222 kB 00:00 2024-05-09T08:52:14.103 INFO:teuthology.orchestra.run.smithi183.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 647 kB/s | 27 kB 00:00 2024-05-09T08:52:14.145 INFO:teuthology.orchestra.run.smithi183.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 763 kB/s | 32 kB 00:00 2024-05-09T08:52:14.245 INFO:teuthology.orchestra.run.smithi183.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 7.3 MB/s | 746 kB 00:00 2024-05-09T08:52:14.304 INFO:teuthology.orchestra.run.smithi183.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.7 MB/s | 163 kB 00:00 2024-05-09T08:52:14.354 INFO:teuthology.orchestra.run.smithi183.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 2.5 MB/s | 127 kB 00:00 2024-05-09T08:52:14.421 INFO:teuthology.orchestra.run.smithi183.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 7.3 MB/s | 500 kB 00:00 2024-05-09T08:52:14.872 INFO:teuthology.orchestra.run.smithi183.stdout:(18/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 12 MB/s | 5.3 MB 00:00 2024-05-09T08:52:14.905 INFO:teuthology.orchestra.run.smithi183.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 1.3 MB/s | 45 kB 00:00 2024-05-09T08:52:14.955 INFO:teuthology.orchestra.run.smithi183.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 2.6 MB/s | 135 kB 00:00 2024-05-09T08:52:15.006 INFO:teuthology.orchestra.run.smithi183.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 3.2 MB/s | 162 kB 00:00 2024-05-09T08:52:15.073 INFO:teuthology.orchestra.run.smithi183.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 4.7 MB/s | 321 kB 00:00 2024-05-09T08:52:15.123 INFO:teuthology.orchestra.run.smithi183.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 5.8 MB/s | 298 kB 00:00 2024-05-09T08:52:15.182 INFO:teuthology.orchestra.run.smithi183.stdout:(24/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T08:52:15.232 INFO:teuthology.orchestra.run.smithi183.stdout:(25/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-09T08:52:15.499 INFO:teuthology.orchestra.run.smithi183.stdout:(26/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 12 MB/s | 3.1 MB 00:00 2024-05-09T08:52:15.558 INFO:teuthology.orchestra.run.smithi183.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-09T08:52:15.600 INFO:teuthology.orchestra.run.smithi183.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 763 kB/s | 32 kB 00:00 2024-05-09T08:52:15.650 INFO:teuthology.orchestra.run.smithi183.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 2.8 MB/s | 142 kB 00:00 2024-05-09T08:52:15.942 INFO:teuthology.orchestra.run.smithi183.stdout:(30/156): ceph-mgr-dashboard-19.0.0-2515.g96805 12 MB/s | 3.5 MB 00:00 2024-05-09T08:52:16.517 INFO:teuthology.orchestra.run.smithi183.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-05-09T08:52:16.567 INFO:teuthology.orchestra.run.smithi183.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 4.9 MB/s | 249 kB 00:00 2024-05-09T08:52:16.618 INFO:teuthology.orchestra.run.smithi183.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 988 kB/s | 49 kB 00:00 2024-05-09T08:52:16.659 INFO:teuthology.orchestra.run.smithi183.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 396 kB/s | 16 kB 00:00 2024-05-09T08:52:16.701 INFO:teuthology.orchestra.run.smithi183.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 6.2 MB/s | 264 kB 00:00 2024-05-09T08:52:16.801 INFO:teuthology.orchestra.run.smithi183.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 7.5 MB/s | 763 kB 00:00 2024-05-09T08:52:16.844 INFO:teuthology.orchestra.run.smithi183.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 438 kB/s | 18 kB 00:00 2024-05-09T08:52:17.077 INFO:teuthology.orchestra.run.smithi183.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 80 kB 00:00 2024-05-09T08:52:17.561 INFO:teuthology.orchestra.run.smithi183.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-09T08:52:17.703 INFO:teuthology.orchestra.run.smithi183.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 195 kB 00:00 2024-05-09T08:52:17.836 INFO:teuthology.orchestra.run.smithi183.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-09T08:52:18.403 INFO:teuthology.orchestra.run.smithi183.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-05-09T08:52:18.478 INFO:teuthology.orchestra.run.smithi183.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-09T08:52:18.554 INFO:teuthology.orchestra.run.smithi183.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-05-09T08:52:18.637 INFO:teuthology.orchestra.run.smithi183.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-09T08:52:18.780 INFO:teuthology.orchestra.run.smithi183.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-05-09T08:52:19.754 INFO:teuthology.orchestra.run.smithi183.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 707 kB/s | 689 kB 00:00 2024-05-09T08:52:20.013 INFO:teuthology.orchestra.run.smithi183.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-05-09T08:52:21.363 INFO:teuthology.orchestra.run.smithi183.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 79 kB/s | 107 kB 00:01 2024-05-09T08:52:21.488 INFO:teuthology.orchestra.run.smithi183.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 172 kB/s | 21 kB 00:00 2024-05-09T08:52:21.614 INFO:teuthology.orchestra.run.smithi183.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-05-09T08:52:21.739 INFO:teuthology.orchestra.run.smithi183.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-05-09T08:52:22.714 INFO:teuthology.orchestra.run.smithi183.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-05-09T08:52:22.848 INFO:teuthology.orchestra.run.smithi183.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 137 kB/s | 18 kB 00:00 2024-05-09T08:52:22.982 INFO:teuthology.orchestra.run.smithi183.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 83 kB/s | 11 kB 00:00 2024-05-09T08:52:23.115 INFO:teuthology.orchestra.run.smithi183.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T08:52:23.257 INFO:teuthology.orchestra.run.smithi183.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-05-09T08:52:23.599 INFO:teuthology.orchestra.run.smithi183.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-05-09T08:52:23.733 INFO:teuthology.orchestra.run.smithi183.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-05-09T08:52:23.858 INFO:teuthology.orchestra.run.smithi183.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-05-09T08:52:23.983 INFO:teuthology.orchestra.run.smithi183.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 380 kB/s | 47 kB 00:00 2024-05-09T08:52:24.134 INFO:teuthology.orchestra.run.smithi183.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-09T08:52:24.267 INFO:teuthology.orchestra.run.smithi183.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-09T08:52:24.392 INFO:teuthology.orchestra.run.smithi183.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 282 kB/s | 35 kB 00:00 2024-05-09T08:52:25.968 INFO:teuthology.orchestra.run.smithi183.stdout:(65/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.0 MB/s | 4.7 MB 00:01 2024-05-09T08:52:32.484 INFO:teuthology.orchestra.run.smithi183.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 695 kB/s | 4.4 MB 00:06 2024-05-09T08:52:33.759 INFO:teuthology.orchestra.run.smithi183.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-05-09T08:52:34.518 INFO:teuthology.orchestra.run.smithi183.stdout:(68/156): ceph-common-19.0.0-2515.g96805dcf.el9 813 kB/s | 21 MB 00:26 2024-05-09T08:52:34.543 INFO:teuthology.orchestra.run.smithi183.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 1.1 MB/s | 911 kB 00:00 2024-05-09T08:52:35.043 INFO:teuthology.orchestra.run.smithi183.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 254 kB/s | 133 kB 00:00 2024-05-09T08:52:35.543 INFO:teuthology.orchestra.run.smithi183.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-05-09T08:52:35.660 INFO:teuthology.orchestra.run.smithi183.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 751 kB/s | 87 kB 00:00 2024-05-09T08:52:36.302 INFO:teuthology.orchestra.run.smithi183.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.3 MB/s | 6.0 MB 00:00 2024-05-09T08:52:36.419 INFO:teuthology.orchestra.run.smithi183.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-09T08:52:36.786 INFO:teuthology.orchestra.run.smithi183.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 676 kB/s | 248 kB 00:00 2024-05-09T08:52:36.886 INFO:teuthology.orchestra.run.smithi183.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-09T08:52:36.987 INFO:teuthology.orchestra.run.smithi183.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-05-09T08:52:37.262 INFO:teuthology.orchestra.run.smithi183.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 626 kB/s | 172 kB 00:00 2024-05-09T08:52:37.371 INFO:teuthology.orchestra.run.smithi183.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-09T08:52:37.538 INFO:teuthology.orchestra.run.smithi183.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:02 2024-05-09T08:52:37.655 INFO:teuthology.orchestra.run.smithi183.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-05-09T08:52:37.863 INFO:teuthology.orchestra.run.smithi183.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-05-09T08:52:37.972 INFO:teuthology.orchestra.run.smithi183.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 713 kB/s | 77 kB 00:00 2024-05-09T08:52:38.081 INFO:teuthology.orchestra.run.smithi183.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 391 kB/s | 42 kB 00:00 2024-05-09T08:52:38.189 INFO:teuthology.orchestra.run.smithi183.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-09T08:52:38.298 INFO:teuthology.orchestra.run.smithi183.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-09T08:52:38.423 INFO:teuthology.orchestra.run.smithi183.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-09T08:52:38.532 INFO:teuthology.orchestra.run.smithi183.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-09T08:52:38.640 INFO:teuthology.orchestra.run.smithi183.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 496 kB/s | 54 kB 00:00 2024-05-09T08:52:41.665 INFO:teuthology.orchestra.run.smithi183.stdout:(90/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 1.7 MB/s | 48 MB 00:27 2024-05-09T08:52:41.741 INFO:teuthology.orchestra.run.smithi183.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.2 MB/s | 5.1 MB 00:04 2024-05-09T08:52:41.882 INFO:teuthology.orchestra.run.smithi183.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 312 kB/s | 44 kB 00:00 2024-05-09T08:52:42.058 INFO:teuthology.orchestra.run.smithi183.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 106 kB/s | 42 kB 00:00 2024-05-09T08:52:42.325 INFO:teuthology.orchestra.run.smithi183.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.4 MB/s | 16 MB 00:03 2024-05-09T08:52:42.350 INFO:teuthology.orchestra.run.smithi183.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 123 kB/s | 36 kB 00:00 2024-05-09T08:52:42.375 INFO:teuthology.orchestra.run.smithi183.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 448 kB/s | 221 kB 00:00 2024-05-09T08:52:42.450 INFO:teuthology.orchestra.run.smithi183.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 378 kB/s | 47 kB 00:00 2024-05-09T08:52:42.501 INFO:teuthology.orchestra.run.smithi183.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-05-09T08:52:42.567 INFO:teuthology.orchestra.run.smithi183.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 802 kB/s | 94 kB 00:00 2024-05-09T08:52:42.684 INFO:teuthology.orchestra.run.smithi183.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-05-09T08:52:42.710 INFO:teuthology.orchestra.run.smithi183.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 305 kB/s | 64 kB 00:00 2024-05-09T08:52:42.743 INFO:teuthology.orchestra.run.smithi183.stdout:(102/156): socat-1.7.4.1-5.el9.x86_64.rpm 775 kB/s | 305 kB 00:00 2024-05-09T08:52:42.852 INFO:teuthology.orchestra.run.smithi183.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-05-09T08:52:43.152 INFO:teuthology.orchestra.run.smithi183.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-05-09T08:52:43.553 INFO:teuthology.orchestra.run.smithi183.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 49 kB/s | 19 kB 00:00 2024-05-09T08:52:43.578 INFO:teuthology.orchestra.run.smithi183.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 425 kB/s | 309 kB 00:00 2024-05-09T08:52:43.728 INFO:teuthology.orchestra.run.smithi183.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-05-09T08:52:43.795 INFO:teuthology.orchestra.run.smithi183.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 521 kB/s | 548 kB 00:01 2024-05-09T08:52:43.871 INFO:teuthology.orchestra.run.smithi183.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-05-09T08:52:43.938 INFO:teuthology.orchestra.run.smithi183.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-05-09T08:52:44.138 INFO:teuthology.orchestra.run.smithi183.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-05-09T08:52:44.638 INFO:teuthology.orchestra.run.smithi183.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 1.1 MB/s | 548 kB 00:00 2024-05-09T08:52:44.697 INFO:teuthology.orchestra.run.smithi183.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.1 MB/s | 839 kB 00:00 2024-05-09T08:52:44.772 INFO:teuthology.orchestra.run.smithi183.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 325 kB/s | 43 kB 00:00 2024-05-09T08:52:44.831 INFO:teuthology.orchestra.run.smithi183.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-05-09T08:52:44.923 INFO:teuthology.orchestra.run.smithi183.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 100 kB/s | 15 kB 00:00 2024-05-09T08:52:45.148 INFO:teuthology.orchestra.run.smithi183.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-05-09T08:52:45.198 INFO:teuthology.orchestra.run.smithi183.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 468 kB/s | 172 kB 00:00 2024-05-09T08:52:45.281 INFO:teuthology.orchestra.run.smithi183.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-05-09T08:52:45.307 INFO:teuthology.orchestra.run.smithi183.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-05-09T08:52:45.332 INFO:teuthology.orchestra.run.smithi183.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 263 kB/s | 35 kB 00:00 2024-05-09T08:52:45.524 INFO:teuthology.orchestra.run.smithi183.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 849 kB/s | 206 kB 00:00 2024-05-09T08:52:45.582 INFO:teuthology.orchestra.run.smithi183.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 578 kB/s | 144 kB 00:00 2024-05-09T08:52:45.650 INFO:teuthology.orchestra.run.smithi183.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-05-09T08:52:45.708 INFO:teuthology.orchestra.run.smithi183.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-05-09T08:52:45.783 INFO:teuthology.orchestra.run.smithi183.stdout:(126/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-05-09T08:52:45.809 INFO:teuthology.orchestra.run.smithi183.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-05-09T08:52:45.842 INFO:teuthology.orchestra.run.smithi183.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-05-09T08:52:45.909 INFO:teuthology.orchestra.run.smithi183.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-05-09T08:52:45.976 INFO:teuthology.orchestra.run.smithi183.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-05-09T08:52:46.043 INFO:teuthology.orchestra.run.smithi183.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-05-09T08:52:46.102 INFO:teuthology.orchestra.run.smithi183.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-05-09T08:52:46.177 INFO:teuthology.orchestra.run.smithi183.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-05-09T08:52:46.219 INFO:teuthology.orchestra.run.smithi183.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-05-09T08:52:46.303 INFO:teuthology.orchestra.run.smithi183.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 717 kB/s | 90 kB 00:00 2024-05-09T08:52:46.353 INFO:teuthology.orchestra.run.smithi183.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 211 kB/s | 28 kB 00:00 2024-05-09T08:52:46.428 INFO:teuthology.orchestra.run.smithi183.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-05-09T08:52:46.554 INFO:teuthology.orchestra.run.smithi183.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-05-09T08:52:46.587 INFO:teuthology.orchestra.run.smithi183.stdout:(139/156): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2024-05-09T08:52:46.612 INFO:teuthology.orchestra.run.smithi183.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 727 kB/s | 188 kB 00:00 2024-05-09T08:52:46.671 INFO:teuthology.orchestra.run.smithi183.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-05-09T08:52:46.722 INFO:teuthology.orchestra.run.smithi183.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 381 kB/s | 51 kB 00:00 2024-05-09T08:52:46.797 INFO:teuthology.orchestra.run.smithi183.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 716 kB/s | 90 kB 00:00 2024-05-09T08:52:46.847 INFO:teuthology.orchestra.run.smithi183.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 981 kB/s | 230 kB 00:00 2024-05-09T08:52:46.939 INFO:teuthology.orchestra.run.smithi183.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-05-09T08:52:47.089 INFO:teuthology.orchestra.run.smithi183.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-05-09T08:52:47.114 INFO:teuthology.orchestra.run.smithi183.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 717 kB/s | 192 kB 00:00 2024-05-09T08:52:47.290 INFO:teuthology.orchestra.run.smithi183.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-05-09T08:52:47.482 INFO:teuthology.orchestra.run.smithi183.stdout:(149/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 8.5 MB/s | 3.1 MB 00:00 2024-05-09T08:52:47.657 INFO:teuthology.orchestra.run.smithi183.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-05-09T08:52:47.815 INFO:teuthology.orchestra.run.smithi183.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 264 kB/s | 42 kB 00:00 2024-05-09T08:52:47.966 INFO:teuthology.orchestra.run.smithi183.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.6 MB/s | 248 kB 00:00 2024-05-09T08:52:48.066 INFO:teuthology.orchestra.run.smithi183.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.1 MB/s | 1.8 MB 00:00 2024-05-09T08:52:48.216 INFO:teuthology.orchestra.run.smithi183.stdout:(154/156): librados2-19.0.0-2515.g96805dcf.el9. 3.0 MB/s | 3.4 MB 00:01 2024-05-09T08:52:48.299 INFO:teuthology.orchestra.run.smithi183.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 1.3 MB 00:00 2024-05-09T08:52:48.825 INFO:teuthology.orchestra.run.smithi183.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 2.4 MB 00:00 2024-05-09T08:52:48.828 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:52:48.828 INFO:teuthology.orchestra.run.smithi183.stdout:Total 5.4 MB/s | 225 MB 00:41 2024-05-09T08:52:49.953 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-09T08:52:50.102 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-09T08:52:50.102 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-09T08:52:51.982 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-09T08:52:51.982 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-09T08:52:58.035 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-09T08:52:58.120 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:52:58.167 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:52:58.221 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:52:58.255 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:52:58.280 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:52:58.323 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:52:58.651 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:52:58.781 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:52:58.902 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:52:58.940 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:52:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:52:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:52:59.213 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:52:59.267 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:52:59.295 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:52:59.398 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:52:59.421 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:52:59.483 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:52:59.505 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:52:59.530 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:52:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:52:59.811 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:52:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:53:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:53:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:53:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:53:00.407 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:53:00.526 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:53:00.614 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:53:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:53:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:53:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:53:00.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:53:00.830 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:53:00.862 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:53:00.890 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:53:00.959 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:53:01.012 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:53:01.043 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:53:01.078 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:53:01.154 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:53:01.189 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:53:01.224 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:53:01.253 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:53:01.324 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:53:01.368 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:53:01.450 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:53:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:53:01.637 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:53:02.168 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:53:02.257 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:53:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:53:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:53:02.359 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:53:02.400 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:53:02.558 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:53:02.612 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:53:02.666 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:53:02.739 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:53:02.770 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:53:02.807 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:53:02.853 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:53:02.890 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:53:02.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:53:02.974 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:53:03.111 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:53:03.155 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:53:03.185 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:53:03.227 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:53:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:53:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:53:03.392 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:53:03.426 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:53:03.451 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:53:03.526 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:53:03.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:53:03.708 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:53:03.736 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:53:03.770 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:53:03.809 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:53:03.841 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:53:03.871 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:53:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:53:03.938 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:53:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:53:04.014 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:53:04.102 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:53:04.145 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:53:04.201 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:53:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:53:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:53:04.403 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:53:05.130 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:53:05.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:53:05.208 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:53:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:53:05.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:53:05.454 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:53:05.485 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:53:05.661 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:53:05.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:53:05.768 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:53:06.420 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:53:06.532 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:53:07.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:53:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:53:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:53:07.907 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:53:08.220 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:53:08.262 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:53:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:53:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:53:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:53:08.883 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:53:08.915 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:53:08.985 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:53:09.008 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:53:11.303 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:53:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:53:11.895 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:53:11.941 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:53:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:53:12.031 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:53:12.031 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:53:12.031 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:12.059 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:53:17.618 INFO:teuthology.orchestra.run.smithi049.stdout:CentOS Stream 9 - Extras packages 199 B/s | 16 kB 01:23 2024-05-09T08:53:19.378 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-09T08:53:23.581 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:53:23.581 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-05-09T08:53:23.581 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-05-09T08:53:23.581 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-05-09T08:53:23.581 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-05-09T08:53:23.582 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-05-09T08:53:23.582 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-05-09T08:53:23.582 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-05-09T08:53:23.582 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:23.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:53:23.760 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:53:23.760 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:23.760 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:53:23.760 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:24.670 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T08:53:25.256 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:53:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:53:25.282 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:25.282 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:53:25.282 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:53:25.282 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:53:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:25.506 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:53:25.528 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:53:25.528 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:25.528 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:53:25.528 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:53:25.528 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:53:25.529 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:25.965 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:53:25.987 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:53:25.987 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:25.987 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:53:25.987 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:53:25.988 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:53:25.988 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:26.087 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:53:26.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:53:26.200 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:53:26.385 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:53:26.433 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:53:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:53:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:53:26.760 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:53:26.816 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:53:26.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:53:26.875 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:53:27.072 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:53:27.072 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T08:53:27.084 INFO:teuthology.orchestra.run.smithi049.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T08:53:27.084 INFO:teuthology.orchestra.run.smithi049.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T08:53:27.085 INFO:teuthology.orchestra.run.smithi049.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T08:53:27.086 INFO:teuthology.orchestra.run.smithi049.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T08:53:27.088 INFO:teuthology.orchestra.run.smithi049.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T08:53:27.090 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T08:53:27.091 INFO:teuthology.orchestra.run.smithi049.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T08:53:27.155 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-05-09T08:53:27.176 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-09T08:53:27.176 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T08:53:27.177 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T08:53:27.178 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T08:53:27.179 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T08:53:27.180 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T08:53:27.181 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T08:53:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T08:53:27.183 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T08:53:27.184 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T08:53:27.185 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T08:53:27.186 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T08:53:27.187 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T08:53:27.188 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout:Installing weak dependencies: 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-05-09T08:53:27.189 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================= 2024-05-09T08:53:27.190 INFO:teuthology.orchestra.run.smithi049.stdout:Install 148 Packages 2024-05-09T08:53:27.190 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 8 Packages 2024-05-09T08:53:27.190 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:53:27.190 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 225 M 2024-05-09T08:53:27.190 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-05-09T08:53:27.248 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:53:27.290 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:53:27.330 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:53:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:53:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:53:28.271 INFO:teuthology.orchestra.run.smithi049.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-05-09T08:53:28.454 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:53:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:53:28.871 INFO:teuthology.orchestra.run.smithi049.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 889 kB 00:00 2024-05-09T08:53:28.997 INFO:teuthology.orchestra.run.smithi049.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.1 MB/s | 145 kB 00:00 2024-05-09T08:53:29.457 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:53:29.482 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:53:29.584 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:53:29.714 INFO:teuthology.orchestra.run.smithi049.stdout:(4/156): ceph-common-19.0.0-2515.g96805dcf.el9. 12 MB/s | 21 MB 00:01 2024-05-09T08:53:29.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:53:29.784 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:53:29.806 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:53:29.806 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:29.806 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:53:29.806 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:53:29.807 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:53:29.807 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:29.839 INFO:teuthology.orchestra.run.smithi049.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 12 MB/s | 1.5 MB 00:00 2024-05-09T08:53:29.859 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:53:29.870 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:53:29.964 INFO:teuthology.orchestra.run.smithi049.stdout:(6/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-05-09T08:53:29.968 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:53:30.014 INFO:teuthology.orchestra.run.smithi049.stdout:(7/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-05-09T08:53:30.017 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:53:30.123 INFO:teuthology.orchestra.run.smithi049.stdout:(8/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 17 MB/s | 4.7 MB 00:00 2024-05-09T08:53:30.156 INFO:teuthology.orchestra.run.smithi049.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 763 kB/s | 25 kB 00:00 2024-05-09T08:53:30.912 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:53:30.945 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:53:30.967 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:53:30.967 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:30.968 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:53:30.968 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:53:30.968 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:53:30.968 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:31.012 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:53:31.034 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:53:31.034 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:31.034 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:53:31.034 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:31.080 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:53:31.101 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:53:31.102 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:31.102 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:53:31.102 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:53:31.102 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:53:31.102 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:31.405 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:53:31.427 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:32.707 INFO:teuthology.orchestra.run.smithi049.stdout:(10/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 3.8 MB/s | 10 MB 00:02 2024-05-09T08:53:32.932 INFO:teuthology.orchestra.run.smithi049.stdout:(11/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x8 5.7 MB/s | 17 MB 00:02 2024-05-09T08:53:32.957 INFO:teuthology.orchestra.run.smithi049.stdout:(12/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 888 kB/s | 222 kB 00:00 2024-05-09T08:53:33.024 INFO:teuthology.orchestra.run.smithi049.stdout:(13/156): cephfs-shell-19.0.0-2515.g96805dcf.el 294 kB/s | 27 kB 00:00 2024-05-09T08:53:33.050 INFO:teuthology.orchestra.run.smithi049.stdout:(14/156): libcephfs-devel-19.0.0-2515.g96805dcf 343 kB/s | 32 kB 00:00 2024-05-09T08:53:33.475 INFO:teuthology.orchestra.run.smithi049.stdout:(15/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 15 MB/s | 48 MB 00:03 2024-05-09T08:53:33.500 INFO:teuthology.orchestra.run.smithi049.stdout:(16/156): libcephsqlite-19.0.0-2515.g96805dcf.e 363 kB/s | 163 kB 00:00 2024-05-09T08:53:33.526 INFO:teuthology.orchestra.run.smithi049.stdout:(17/156): librados-devel-19.0.0-2515.g96805dcf. 2.5 MB/s | 127 kB 00:00 2024-05-09T08:53:33.559 INFO:teuthology.orchestra.run.smithi049.stdout:(18/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 1.4 MB/s | 746 kB 00:00 2024-05-09T08:53:33.643 INFO:teuthology.orchestra.run.smithi049.stdout:(19/156): libradosstriper1-19.0.0-2515.g96805dc 3.4 MB/s | 500 kB 00:00 2024-05-09T08:53:33.672 INFO:teuthology.orchestra.run.smithi049.stdout:(20/156): python3-ceph-argparse-19.0.0-2515.g96 416 kB/s | 45 kB 00:00 2024-05-09T08:53:33.743 INFO:teuthology.orchestra.run.smithi049.stdout:(21/156): python3-ceph-common-19.0.0-2515.g9680 1.3 MB/s | 135 kB 00:00 2024-05-09T08:53:33.777 INFO:teuthology.orchestra.run.smithi049.stdout:(22/156): python3-cephfs-19.0.0-2515.g96805dcf. 1.5 MB/s | 162 kB 00:00 2024-05-09T08:53:33.861 INFO:teuthology.orchestra.run.smithi049.stdout:(23/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 16 MB/s | 5.3 MB 00:00 2024-05-09T08:53:33.886 INFO:teuthology.orchestra.run.smithi049.stdout:(24/156): python3-rados-19.0.0-2515.g96805dcf.e 2.2 MB/s | 321 kB 00:00 2024-05-09T08:53:33.911 INFO:teuthology.orchestra.run.smithi049.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.9 MB/s | 99 kB 00:00 2024-05-09T08:53:33.936 INFO:teuthology.orchestra.run.smithi049.stdout:(26/156): python3-rbd-19.0.0-2515.g96805dcf.el9 1.8 MB/s | 298 kB 00:00 2024-05-09T08:53:33.978 INFO:teuthology.orchestra.run.smithi049.stdout:(27/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 927 kB/s | 85 kB 00:00 2024-05-09T08:53:34.028 INFO:teuthology.orchestra.run.smithi049.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-09T08:53:34.062 INFO:teuthology.orchestra.run.smithi049.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 380 kB/s | 32 kB 00:00 2024-05-09T08:53:34.129 INFO:teuthology.orchestra.run.smithi049.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 14 MB/s | 3.1 MB 00:00 2024-05-09T08:53:34.154 INFO:teuthology.orchestra.run.smithi049.stdout:(31/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.1 MB/s | 142 kB 00:00 2024-05-09T08:53:34.246 INFO:teuthology.orchestra.run.smithi049.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.7 MB/s | 249 kB 00:00 2024-05-09T08:53:34.330 INFO:teuthology.orchestra.run.smithi049.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 593 kB/s | 49 kB 00:00 2024-05-09T08:53:34.431 INFO:teuthology.orchestra.run.smithi049.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 164 kB/s | 16 kB 00:00 2024-05-09T08:53:34.572 INFO:teuthology.orchestra.run.smithi049.stdout:(35/156): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-05-09T08:53:34.597 INFO:teuthology.orchestra.run.smithi049.stdout:(36/156): ceph-volume-19.0.0-2515.g96805dcf.el9 1.5 MB/s | 264 kB 00:00 2024-05-09T08:53:34.648 INFO:teuthology.orchestra.run.smithi049.stdout:(37/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 9.9 MB/s | 763 kB 00:00 2024-05-09T08:53:34.698 INFO:teuthology.orchestra.run.smithi049.stdout:(38/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 182 kB/s | 18 kB 00:00 2024-05-09T08:53:34.798 INFO:teuthology.orchestra.run.smithi049.stdout:(39/156): ceph-mgr-dashboard-19.0.0-2515.g96805 4.8 MB/s | 3.5 MB 00:00 2024-05-09T08:53:34.824 INFO:teuthology.orchestra.run.smithi049.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 458 kB/s | 80 kB 00:00 2024-05-09T08:53:35.066 INFO:teuthology.orchestra.run.smithi049.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-05-09T08:53:35.108 INFO:teuthology.orchestra.run.smithi049.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 632 kB/s | 195 kB 00:00 2024-05-09T08:53:35.216 INFO:teuthology.orchestra.run.smithi049.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-05-09T08:53:35.292 INFO:teuthology.orchestra.run.smithi049.stdout:(44/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-05-09T08:53:35.334 INFO:teuthology.orchestra.run.smithi049.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-09T08:53:35.409 INFO:teuthology.orchestra.run.smithi049.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-09T08:53:35.434 INFO:teuthology.orchestra.run.smithi049.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-09T08:53:35.718 INFO:teuthology.orchestra.run.smithi049.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 1.9 MB/s | 1.2 MB 00:00 2024-05-09T08:53:35.917 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:53:35.966 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:53:36.008 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:53:36.040 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:53:36.067 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:53:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:53:36.304 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:53:36.335 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:53:36.364 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:53:36.419 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:53:36.651 INFO:teuthology.orchestra.run.smithi049.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 555 kB/s | 689 kB 00:01 2024-05-09T08:53:36.677 INFO:teuthology.orchestra.run.smithi049.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 118 kB/s | 107 kB 00:00 2024-05-09T08:53:36.810 INFO:teuthology.orchestra.run.smithi049.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 686 kB/s | 943 kB 00:01 2024-05-09T08:53:37.063 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:53:37.108 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:53:37.127 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:53:37.377 INFO:teuthology.orchestra.run.smithi049.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 30 kB/s | 21 kB 00:00 2024-05-09T08:53:37.527 INFO:teuthology.orchestra.run.smithi049.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 46 kB/s | 33 kB 00:00 2024-05-09T08:53:37.728 INFO:teuthology.orchestra.run.smithi049.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 298 kB/s | 313 kB 00:01 2024-05-09T08:53:37.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:53:38.062 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:53:38.112 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:53:38.203 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:53:38.228 INFO:teuthology.orchestra.run.smithi049.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 26 kB/s | 18 kB 00:00 2024-05-09T08:53:38.248 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:53:38.275 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:53:38.275 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:53:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:53:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:53:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:53:38.420 INFO:teuthology.orchestra.run.smithi049.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 16 kB/s | 11 kB 00:00 2024-05-09T08:53:38.484 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:53:38.509 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:53:38.514 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:53:38.514 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:53:38.694 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:53:38.695 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:53:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:53:38.850 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:53:38.862 INFO:teuthology.orchestra.run.smithi049.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.0 MB/s | 3.0 MB 00:01 2024-05-09T08:53:39.245 INFO:teuthology.orchestra.run.smithi049.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 55 kB/s | 45 kB 00:00 2024-05-09T08:53:39.337 INFO:teuthology.orchestra.run.smithi049.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 144 kB/s | 160 kB 00:01 2024-05-09T08:53:40.079 INFO:teuthology.orchestra.run.smithi049.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 545 kB/s | 662 kB 00:01 2024-05-09T08:53:40.104 INFO:teuthology.orchestra.run.smithi049.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 33 kB/s | 25 kB 00:00 2024-05-09T08:53:40.429 INFO:teuthology.orchestra.run.smithi049.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 492 kB/s | 582 kB 00:01 2024-05-09T08:53:40.871 INFO:teuthology.orchestra.run.smithi049.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 60 kB/s | 47 kB 00:00 2024-05-09T08:53:41.172 INFO:teuthology.orchestra.run.smithi049.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 274 kB/s | 292 kB 00:01 2024-05-09T08:53:41.414 INFO:teuthology.orchestra.run.smithi049.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 192 kB/s | 188 kB 00:00 2024-05-09T08:53:41.622 INFO:teuthology.orchestra.run.smithi049.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 47 kB/s | 35 kB 00:00 2024-05-09T08:53:42.514 INFO:teuthology.orchestra.run.smithi049.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-05-09T08:53:42.774 INFO:teuthology.orchestra.run.smithi049.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-05-09T08:53:42.848 INFO:teuthology.orchestra.run.smithi049.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-05-09T08:53:43.765 INFO:teuthology.orchestra.run.smithi049.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 134 kB/s | 133 kB 00:00 2024-05-09T08:53:43.874 INFO:teuthology.orchestra.run.smithi049.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 670 kB/s | 911 kB 00:01 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:53:44.271 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:53:44.272 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:53:44.273 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:53:44.274 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:53:44.275 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:53:44.276 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:53:44.277 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:53:44.278 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:53:44.279 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:53:44.280 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:53:44.281 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:53:44.282 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:53:44.283 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:53:44.599 INFO:teuthology.orchestra.run.smithi049.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-05-09T08:53:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 90 kB/s | 87 kB 00:00 2024-05-09T08:53:45.149 INFO:teuthology.orchestra.run.smithi049.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 747 kB/s | 1.0 MB 00:01 2024-05-09T08:53:45.974 INFO:teuthology.orchestra.run.smithi049.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 217 kB/s | 244 kB 00:01 2024-05-09T08:53:46.283 INFO:teuthology.orchestra.run.smithi049.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 224 kB/s | 248 kB 00:01 2024-05-09T08:53:46.392 INFO:teuthology.orchestra.run.smithi049.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-05-09T08:53:46.750 INFO:teuthology.orchestra.run.smithi049.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 33 kB/s | 26 kB 00:00 2024-05-09T08:53:47.034 INFO:teuthology.orchestra.run.smithi049.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 27 kB/s | 19 kB 00:00 2024-05-09T08:53:47.124 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:53:47.124 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:47.124 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.125 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.126 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.127 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:53:47.128 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:53:47.129 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:53:47.130 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:53:47.131 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:53:47.132 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:53:47.133 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:53:47.134 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:53:47.135 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:53:47.135 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:53:47.135 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.135 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.135 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:53:47.136 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:53:47.137 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:53:47.138 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:53:47.139 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:53:47.139 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:53:47.139 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-09T08:53:47.459 INFO:teuthology.orchestra.run.smithi049.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 161 kB/s | 172 kB 00:01 2024-05-09T08:53:47.526 DEBUG:teuthology.parallel:result is None 2024-05-09T08:53:47.584 INFO:teuthology.orchestra.run.smithi049.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 42 kB/s | 35 kB 00:00 2024-05-09T08:53:48.626 INFO:teuthology.orchestra.run.smithi049.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 215 kB/s | 250 kB 00:01 2024-05-09T08:53:48.702 INFO:teuthology.orchestra.run.smithi049.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 199 kB/s | 222 kB 00:01 2024-05-09T08:53:48.868 INFO:teuthology.orchestra.run.smithi049.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-05-09T08:53:49.535 INFO:teuthology.orchestra.run.smithi049.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 51 kB/s | 42 kB 00:00 2024-05-09T08:53:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 80 kB/s | 77 kB 00:00 2024-05-09T08:53:50.202 INFO:teuthology.orchestra.run.smithi049.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 202 kB/s | 269 kB 00:01 2024-05-09T08:53:50.636 INFO:teuthology.orchestra.run.smithi049.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 144 kB/s | 159 kB 00:01 2024-05-09T08:53:50.728 INFO:teuthology.orchestra.run.smithi049.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 246 kB/s | 279 kB 00:01 2024-05-09T08:53:51.111 INFO:teuthology.orchestra.run.smithi049.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 56 kB/s | 51 kB 00:00 2024-05-09T08:53:51.562 INFO:teuthology.orchestra.run.smithi049.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 58 kB/s | 54 kB 00:00 2024-05-09T08:53:51.962 INFO:teuthology.orchestra.run.smithi049.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 49 kB/s | 42 kB 00:00 2024-05-09T08:53:52.429 INFO:teuthology.orchestra.run.smithi049.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 51 kB/s | 44 kB 00:00 2024-05-09T08:53:52.946 INFO:teuthology.orchestra.run.smithi049.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-05-09T08:53:53.071 INFO:teuthology.orchestra.run.smithi049.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 199 kB/s | 221 kB 00:01 2024-05-09T08:53:53.304 INFO:teuthology.orchestra.run.smithi049.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 41 kB/s | 36 kB 00:00 2024-05-09T08:53:53.747 INFO:teuthology.orchestra.run.smithi049.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 59 kB/s | 47 kB 00:00 2024-05-09T08:53:54.188 INFO:teuthology.orchestra.run.smithi049.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 273 kB/s | 305 kB 00:01 2024-05-09T08:53:54.214 INFO:teuthology.orchestra.run.smithi049.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 64 kB/s | 58 kB 00:00 2024-05-09T08:53:54.697 INFO:teuthology.orchestra.run.smithi049.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 99 kB/s | 94 kB 00:00 2024-05-09T08:53:54.834 INFO:teuthology.orchestra.run.smithi049.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 168 kB/s | 22 kB 00:00 2024-05-09T08:53:54.981 INFO:teuthology.orchestra.run.smithi049.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 58 kB/s | 45 kB 00:00 2024-05-09T08:53:55.015 INFO:teuthology.orchestra.run.smithi049.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 77 kB/s | 64 kB 00:00 2024-05-09T08:53:55.107 INFO:teuthology.orchestra.run.smithi049.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-05-09T08:53:55.411 INFO:teuthology.orchestra.run.smithi049.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 64 kB/s | 19 kB 00:00 2024-05-09T08:53:55.641 INFO:teuthology.orchestra.run.smithi049.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 494 kB/s | 309 kB 00:00 2024-05-09T08:53:55.933 INFO:teuthology.orchestra.run.smithi049.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 576 kB/s | 548 kB 00:00 2024-05-09T08:53:56.258 INFO:teuthology.orchestra.run.smithi049.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 42 kB/s | 26 kB 00:00 2024-05-09T08:53:56.292 INFO:teuthology.orchestra.run.smithi049.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 135 kB/s | 49 kB 00:00 2024-05-09T08:53:56.633 INFO:teuthology.orchestra.run.smithi049.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 180 kB/s | 67 kB 00:00 2024-05-09T08:53:56.759 INFO:teuthology.orchestra.run.smithi049.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 324 kB/s | 151 kB 00:00 2024-05-09T08:53:57.784 INFO:teuthology.orchestra.run.smithi049.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 535 kB/s | 548 kB 00:01 2024-05-09T08:53:57.926 INFO:teuthology.orchestra.run.smithi049.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 649 kB/s | 839 kB 00:01 2024-05-09T08:53:58.209 INFO:teuthology.orchestra.run.smithi049.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 114 kB/s | 32 kB 00:00 2024-05-09T08:53:58.235 INFO:teuthology.orchestra.run.smithi049.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 96 kB/s | 43 kB 00:00 2024-05-09T08:53:58.560 INFO:teuthology.orchestra.run.smithi049.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 43 kB/s | 15 kB 00:00 2024-05-09T08:53:58.794 INFO:teuthology.orchestra.run.smithi049.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 307 kB/s | 172 kB 00:00 2024-05-09T08:53:59.369 INFO:teuthology.orchestra.run.smithi049.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 443 kB/s | 358 kB 00:00 2024-05-09T08:53:59.494 INFO:teuthology.orchestra.run.smithi049.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 433 kB/s | 303 kB 00:00 2024-05-09T08:53:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 121 kB/s | 35 kB 00:00 2024-05-09T08:54:00.028 INFO:teuthology.orchestra.run.smithi049.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 979 kB/s | 4.4 MB 00:04 2024-05-09T08:54:00.145 INFO:teuthology.orchestra.run.smithi049.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 316 kB/s | 206 kB 00:00 2024-05-09T08:54:00.370 INFO:teuthology.orchestra.run.smithi049.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 47 kB/s | 11 kB 00:00 2024-05-09T08:54:00.571 INFO:teuthology.orchestra.run.smithi049.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 266 kB/s | 144 kB 00:00 2024-05-09T08:54:00.704 INFO:teuthology.orchestra.run.smithi049.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 53 kB/s | 18 kB 00:00 2024-05-09T08:54:00.880 INFO:teuthology.orchestra.run.smithi049.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 75 kB/s | 23 kB 00:00 2024-05-09T08:54:01.005 INFO:teuthology.orchestra.run.smithi049.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 65 kB/s | 19 kB 00:00 2024-05-09T08:54:01.172 INFO:teuthology.orchestra.run.smithi049.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 67 kB/s | 20 kB 00:00 2024-05-09T08:54:01.622 INFO:teuthology.orchestra.run.smithi049.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 103 kB/s | 46 kB 00:00 2024-05-09T08:54:02.081 INFO:teuthology.orchestra.run.smithi049.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 862 kB/s | 2.0 MB 00:02 2024-05-09T08:54:02.106 INFO:teuthology.orchestra.run.smithi049.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 163 kB/s | 79 kB 00:00 2024-05-09T08:54:02.431 INFO:teuthology.orchestra.run.smithi049.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-05-09T08:54:02.707 INFO:teuthology.orchestra.run.smithi049.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 454 kB/s | 272 kB 00:00 2024-05-09T08:54:02.732 INFO:teuthology.orchestra.run.smithi049.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 55 kB/s | 16 kB 00:00 2024-05-09T08:54:02.765 INFO:teuthology.orchestra.run.smithi049.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 594 kB/s | 1.0 MB 00:01 2024-05-09T08:54:03.084 INFO:teuthology.orchestra.run.smithi049.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 80 kB/s | 28 kB 00:00 2024-05-09T08:54:03.108 INFO:teuthology.orchestra.run.smithi049.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 90 kB/s | 31 kB 00:00 2024-05-09T08:54:03.133 INFO:teuthology.orchestra.run.smithi049.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-05-09T08:54:03.416 INFO:teuthology.orchestra.run.smithi049.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 126 kB/s | 36 kB 00:00 2024-05-09T08:54:03.500 INFO:teuthology.orchestra.run.smithi049.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 151 kB/s | 59 kB 00:00 2024-05-09T08:54:03.667 INFO:teuthology.orchestra.run.smithi049.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 324 kB/s | 188 kB 00:00 2024-05-09T08:54:03.834 INFO:teuthology.orchestra.run.smithi049.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 122 kB/s | 51 kB 00:00 2024-05-09T08:54:04.084 INFO:teuthology.orchestra.run.smithi049.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 215 kB/s | 90 kB 00:00 2024-05-09T08:54:04.376 INFO:teuthology.orchestra.run.smithi049.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 68 kB/s | 20 kB 00:00 2024-05-09T08:54:04.618 INFO:teuthology.orchestra.run.smithi049.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 205 kB/s | 230 kB 00:01 2024-05-09T08:54:04.685 INFO:teuthology.orchestra.run.smithi049.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 502 kB/s | 427 kB 00:00 2024-05-09T08:54:04.952 INFO:teuthology.orchestra.run.smithi049.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 333 kB/s | 192 kB 00:00 2024-05-09T08:54:05.344 INFO:teuthology.orchestra.run.smithi049.stdout:(148/156): librados2-19.0.0-2515.g96805dcf.el9. 5.1 MB/s | 3.4 MB 00:00 2024-05-09T08:54:05.536 INFO:teuthology.orchestra.run.smithi049.stdout:(149/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-05-09T08:54:06.169 INFO:teuthology.orchestra.run.smithi049.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.8 MB/s | 2.3 MB 00:00 2024-05-09T08:54:06.245 INFO:teuthology.orchestra.run.smithi049.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 557 kB/s | 42 kB 00:00 2024-05-09T08:54:06.387 INFO:teuthology.orchestra.run.smithi049.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-05-09T08:54:06.429 INFO:teuthology.orchestra.run.smithi049.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-05-09T08:54:06.613 INFO:teuthology.orchestra.run.smithi049.stdout:(154/156): thrift-0.15.0-2.el9.x86_64.rpm 812 kB/s | 1.6 MB 00:01 2024-05-09T08:54:06.788 INFO:teuthology.orchestra.run.smithi049.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-05-09T08:54:07.203 INFO:teuthology.orchestra.run.smithi049.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 2.4 MB 00:00 2024-05-09T08:54:07.206 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-05-09T08:54:07.206 INFO:teuthology.orchestra.run.smithi049.stdout:Total 5.6 MB/s | 225 MB 00:40 2024-05-09T08:54:08.380 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-05-09T08:54:08.550 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-05-09T08:54:08.550 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-05-09T08:54:10.826 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-05-09T08:54:10.827 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-05-09T08:54:17.009 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-05-09T08:54:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T08:54:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T08:54:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T08:54:17.252 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T08:54:17.277 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T08:54:17.337 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T08:54:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T08:54:17.794 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:54:17.927 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:54:17.962 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:54:18.055 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:54:18.083 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:54:18.264 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T08:54:18.328 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T08:54:18.358 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:54:18.447 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:54:18.476 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:54:18.522 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T08:54:18.552 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:54:18.571 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T08:54:18.764 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:54:18.966 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T08:54:19.093 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:54:19.146 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T08:54:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:54:19.346 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:54:19.554 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T08:54:19.656 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T08:54:19.759 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T08:54:19.809 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T08:54:19.846 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T08:54:19.921 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T08:54:19.962 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T08:54:20.005 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T08:54:20.038 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T08:54:20.065 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T08:54:20.139 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T08:54:20.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T08:54:20.226 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T08:54:20.262 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T08:54:20.349 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T08:54:20.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T08:54:20.434 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:54:20.461 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T08:54:20.535 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T08:54:20.585 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T08:54:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T08:54:20.800 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T08:54:20.855 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T08:54:21.407 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T08:54:21.511 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T08:54:21.545 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T08:54:21.589 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T08:54:21.618 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T08:54:21.660 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T08:54:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T08:54:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T08:54:21.944 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T08:54:22.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T08:54:22.055 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T08:54:22.092 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T08:54:22.139 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T08:54:22.174 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T08:54:22.218 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T08:54:22.260 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T08:54:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T08:54:22.449 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T08:54:22.479 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T08:54:22.521 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T08:54:22.626 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T08:54:22.662 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T08:54:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T08:54:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T08:54:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T08:54:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T08:54:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T08:54:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T08:54:23.062 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T08:54:23.098 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T08:54:23.137 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T08:54:23.176 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T08:54:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T08:54:23.240 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T08:54:23.274 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T08:54:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T08:54:23.350 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T08:54:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T08:54:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T08:54:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T08:54:23.583 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T08:54:23.695 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T08:54:23.748 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T08:54:24.511 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T08:54:24.561 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T08:54:24.595 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T08:54:24.697 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T08:54:24.761 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T08:54:24.852 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T08:54:24.888 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T08:54:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T08:54:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T08:54:25.192 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T08:54:25.876 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T08:54:26.004 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T08:54:27.333 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T08:54:27.407 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T08:54:27.435 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T08:54:27.468 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T08:54:27.800 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T08:54:27.839 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:54:27.914 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T08:54:27.944 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T08:54:27.981 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T08:54:28.481 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T08:54:28.509 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:54:28.580 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T08:54:28.610 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T08:54:30.992 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:54:31.024 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:54:31.596 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T08:54:31.635 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:54:31.735 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T08:54:31.735 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T08:54:31.735 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T08:54:31.735 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:31.795 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:54:43.882 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /root 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-05-09T08:54:43.883 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:44.051 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:54:44.067 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T08:54:44.067 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:44.068 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T08:54:44.068 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:45.606 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:54:45.633 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T08:54:45.634 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:45.634 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T08:54:45.634 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:54:45.634 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T08:54:45.634 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:45.873 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T08:54:45.897 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:46.365 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:54:46.389 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T08:54:46.389 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:46.389 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T08:54:46.389 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:54:46.389 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T08:54:46.390 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:46.496 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T08:54:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T08:54:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T08:54:46.815 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T08:54:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T08:54:46.957 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T08:54:47.048 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T08:54:47.191 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T08:54:47.235 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T08:54:47.274 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T08:54:47.303 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T08:54:47.780 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:54:47.810 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T08:54:47.833 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T08:54:47.897 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T08:54:47.924 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:54:49.011 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T08:54:49.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:54:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T08:54:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:54:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T08:54:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T08:54:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:54:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T08:54:50.454 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:50.454 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T08:54:50.454 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:54:50.454 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T08:54:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:54:50.474 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T08:54:50.575 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T08:54:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T08:54:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T08:54:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:54:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T08:54:51.639 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:51.639 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T08:54:51.639 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:54:51.639 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T08:54:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:54:51.705 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T08:54:51.705 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:51.705 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T08:54:51.706 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:51.757 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T08:54:51.782 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:52.108 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:54:52.132 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T08:54:52.132 INFO:teuthology.orchestra.run.smithi049.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T08:54:52.132 INFO:teuthology.orchestra.run.smithi049.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T08:54:52.133 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:54:52.133 INFO:teuthology.orchestra.run.smithi049.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T08:54:52.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:54:56.912 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T08:54:56.954 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T08:54:56.996 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T08:54:57.026 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T08:54:57.053 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T08:54:57.259 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T08:54:57.299 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T08:54:57.329 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T08:54:57.359 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T08:54:57.405 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T08:54:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:54:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:54:58.065 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T08:54:58.977 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T08:54:59.065 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T08:54:59.118 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T08:54:59.210 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T08:54:59.243 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T08:54:59.269 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T08:54:59.269 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:54:59.354 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T08:54:59.387 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T08:54:59.387 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:54:59.471 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T08:54:59.505 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T08:54:59.510 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:54:59.510 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:54:59.699 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T08:54:59.699 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:54:59.782 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T08:54:59.851 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:55:05.280 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T08:55:05.281 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T08:55:05.282 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T08:55:05.283 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T08:55:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T08:55:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T08:55:05.295 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T08:55:05.296 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T08:55:05.297 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T08:55:05.298 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T08:55:05.299 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T08:55:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T08:55:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T08:55:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T08:55:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T08:55:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T08:55:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T08:55:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T08:55:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T08:55:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.106 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.107 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.108 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T08:55:08.109 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.110 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T08:55:08.111 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.112 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T08:55:08.113 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T08:55:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T08:55:08.115 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T08:55:08.116 INFO:teuthology.orchestra.run.smithi049.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T08:55:08.117 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T08:55:08.118 INFO:teuthology.orchestra.run.smithi049.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T08:55:08.119 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:08.119 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-05-09T08:55:08.518 DEBUG:teuthology.parallel:result is None 2024-05-09T08:55:08.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:55:08.662 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:55:08.686 INFO:teuthology.orchestra.run.smithi049.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:55:08.687 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:55:08.688 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:55:08.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T08:55:08.971 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T08:55:08.997 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T08:55:08.998 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T08:55:08.998 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T08:55:09.000 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T08:55:09.000 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:09.000 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:55:09.031 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:09.031 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T08:55:09.067 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T08:55:09.067 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:09.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:55:09.101 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:55:09.169 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:09.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T08:55:09.196 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T08:55:09.262 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T08:55:09.262 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:09.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:55:09.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:55:09.357 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:09.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T08:55:09.383 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T08:55:09.448 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T08:55:09.448 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:09.448 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:55:09.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:55:09.541 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:09.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T08:55:09.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T08:55:09.631 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T08:55:09.711 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T08:55:09.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2024-05-09T08:55:09.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2024-05-09T08:55:09.738 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T08:55:09.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T08:55:09.780 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T08:55:09.804 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T08:55:09.805 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T08:55:09.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T08:55:09.877 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T08:55:09.877 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T08:55:09.878 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T08:55:09.878 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:09.878 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T08:55:09.927 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:10.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:10.066 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:10.066 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T08:55:10.081 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:10.190 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:10.235 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T08:55:10.235 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T08:55:10.235 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7698706', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2334, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '359/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGF0Tu/xeTs7l2x4Qq0xXvIlVrG7D3wHuFlpNwwfRA2iHoSmimelUq72doWmhRYzkhnFxjQm+q0E8mkCrDShKU=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6djY6K06KwtlHhYdcna2V352MkN2KWcUsG5mzROcElNCcx7I6VjhjiGEt4UpIny1Acj2Gk28+l26DFBD9XnPM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['fs/misc/trivial_sync.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_21:24:04', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T08:55:10.236 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T08:55:10.238 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T08:55:10.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T08:55:10.294 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T08:55:10.348 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:10.349 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:55:10.374 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:55:10.374 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-05-09T08:55:10.431 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:55:10.431 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:10.431 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-09 08:55:00.227849229 +0000 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-09 08:51:00.170996914 +0000 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-09 08:51:00.170996914 +0000 2024-05-09T08:55:10.432 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-09 08:51:00.170996914 +0000 2024-05-09T08:55:10.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:55:10.495 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-09T08:55:10.495 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-09T08:55:10.495 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000101822 s, 5.0 MB/s 2024-05-09T08:55:10.497 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:55:10.553 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-05-09T08:55:10.609 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-09 08:55:00.228849204 +0000 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-09 08:51:00.504988127 +0000 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-09 08:51:00.504988127 +0000 2024-05-09T08:55:10.610 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-09 08:51:00.504988127 +0000 2024-05-09T08:55:10.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:55:10.674 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-09T08:55:10.674 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-09T08:55:10.674 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.8807e-05 s, 5.2 MB/s 2024-05-09T08:55:10.675 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:55:10.732 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-09 08:55:00.228849204 +0000 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-09 08:51:00.819979840 +0000 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-09 08:51:00.819979840 +0000 2024-05-09T08:55:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-09 08:51:00.819979840 +0000 2024-05-09T08:55:10.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:55:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-09T08:55:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-09T08:55:10.853 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 8.39e-05 s, 6.1 MB/s 2024-05-09T08:55:10.854 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:55:10.911 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-05-09T08:55:10.967 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-05-09 08:55:00.228849204 +0000 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-09 08:51:01.170970606 +0000 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-09 08:51:01.170970606 +0000 2024-05-09T08:55:10.968 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-09 08:51:01.170970606 +0000 2024-05-09T08:55:10.968 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:55:11.032 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-05-09T08:55:11.032 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-05-09T08:55:11.032 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.9345e-05 s, 5.2 MB/s 2024-05-09T08:55:11.033 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:55:11.090 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:11.090 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:11.091 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T08:55:11.108 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T08:55:11.108 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-05-09T08:55:11.166 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T08:55:11.166 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:11.166 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-09 08:53:39.201339340 +0000 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-09 08:51:13.028613952 +0000 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-09 08:51:13.028613952 +0000 2024-05-09T08:55:11.167 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-09 08:51:13.028613952 +0000 2024-05-09T08:55:11.167 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T08:55:11.232 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-09T08:55:11.232 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-09T08:55:11.232 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00029313 s, 1.7 MB/s 2024-05-09T08:55:11.234 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T08:55:11.291 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-09 08:53:39.201339340 +0000 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-09 08:51:13.373604410 +0000 2024-05-09T08:55:11.348 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-09 08:51:13.373604410 +0000 2024-05-09T08:55:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-09 08:51:13.373604410 +0000 2024-05-09T08:55:11.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T08:55:11.412 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-09T08:55:11.412 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-09T08:55:11.412 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000203229 s, 2.5 MB/s 2024-05-09T08:55:11.413 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T08:55:11.470 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-05-09T08:55:11.525 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T08:55:11.525 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:11.525 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-09 08:53:39.201339340 +0000 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-09 08:51:13.713595007 +0000 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-09 08:51:13.713595007 +0000 2024-05-09T08:55:11.526 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-09 08:51:13.713595007 +0000 2024-05-09T08:55:11.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T08:55:11.589 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-09T08:55:11.589 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-09T08:55:11.589 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000216155 s, 2.4 MB/s 2024-05-09T08:55:11.590 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T08:55:11.646 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-09 08:53:39.202339312 +0000 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-09 08:51:14.069585161 +0000 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-09 08:51:14.069585161 +0000 2024-05-09T08:55:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-09 08:51:14.069585161 +0000 2024-05-09T08:55:11.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T08:55:11.763 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-09T08:55:11.763 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-09T08:55:11.763 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000185701 s, 2.8 MB/s 2024-05-09T08:55:11.764 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T08:55:11.820 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:11.820 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T08:55:11.820 INFO:tasks.ceph:Generating config... 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T08:55:11.822 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T08:55:11.822 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-09T08:55:11.822 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T08:55:11.822 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T08:55:11.822 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T08:55:11.822 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-05-09T08:55:11.823 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T08:55:11.824 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T08:55:11.824 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T08:55:11.825 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T08:55:11.826 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T08:55:11.826 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T08:55:11.827 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T08:55:11.882 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T08:55:11.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T08:55:11.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:55:11.956 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.b', '172.21.15.183'), ('mon.c', '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]')] 2024-05-09T08:55:11.957 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T08:55:11.958 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:11.958 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T08:55:12.013 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --add b 172.21.15.183 --addv c '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid aab8b2de-6010-439c-b8fd-8bc72445265c 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = quincy 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:fsid aab8b2de-6010-439c-b8fd-8bc72445265c 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2024-05-09T08:55:12.102927+0000 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-09T08:55:12.102927+0000 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 17 (quincy) 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-05-09T08:55:12.101 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] mon.c 2024-05-09T08:55:12.102 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T08:55:12.103 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T08:55:12.118 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aab8b2de-6010-439c-b8fd-8bc72445265c... 2024-05-09T08:55:12.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T08:55:12.161 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T08:55:12.203 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2024-05-09T08:55:12.203 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2024-05-09T08:55:12.203 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T08:55:12.203 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.204 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2024-05-09T08:55:12.205 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2024-05-09T08:55:12.206 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.49,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on skipped message = False 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = aab8b2de-6010-439c-b8fd-8bc72445265c 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2024-05-09T08:55:12.207 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug trim objects = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2024-05-09T08:55:12.208 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bdev async discard = True 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bdev enable discard = True 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore allocator = bitmap 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 1/20 2024-05-09T08:55:12.209 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 1/20 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 4/10 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay max = 1 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay probability = 0.005 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay type = client mds 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 2500 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: osd blocked scrub grace period = 3600 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2024-05-09T08:55:12.210 INFO:teuthology.orchestra.run.smithi183.stdout: osd max markdown count = 1000 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.211 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T08:55:12.212 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon down mkfs grace = 300 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: mon op complaint time = 120 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-09T08:55:12.213 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: client mount timeout = 600 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: client_shutdown_timeout = 120 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2024-05-09T08:55:12.214 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.e] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.f] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout:[mds] 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds = 20 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds balancer = 20 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout: mds bal fragment size max = 10000 2024-05-09T08:55:12.215 INFO:teuthology.orchestra.run.smithi183.stdout: mds bal merge size = 5 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds bal split bits = 3 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds bal split size = 100 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug frag = True 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug scatterstat = True 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds oft prefetch dirfrags = True 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds op complaint time = 180 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds verify scatter = True 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds_dir_prefetch = True 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mds_sleep_rank_change = 5000000.0 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay max = 1 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay probability = 0.005 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject delay type = client mds osd 2024-05-09T08:55:12.216 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 2500 2024-05-09T08:55:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2024-05-09T08:55:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2024-05-09T08:55:12.217 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.218 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.219 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T08:55:12.220 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2024-05-09T08:55:12.221 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on skipped message = False 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = aab8b2de-6010-439c-b8fd-8bc72445265c 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2024-05-09T08:55:12.222 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug trim objects = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.223 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bdev async discard = True 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bdev enable discard = True 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore allocator = bitmap 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2024-05-09T08:55:12.224 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: mon client directed command retry = 5 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay max = 1 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay probability = 0.005 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay type = client mds 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject socket failures = 2500 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: osd blocked scrub grace period = 3600 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: osd max markdown count = 1000 2024-05-09T08:55:12.225 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.226 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T08:55:12.227 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon down mkfs grace = 300 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: mon op complaint time = 120 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:12.228 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: client mount timeout = 600 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: client_shutdown_timeout = 120 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: debug client = 20 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2024-05-09T08:55:12.229 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout:[mds] 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds = 20 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: debug mds balancer = 20 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: mds bal fragment size max = 10000 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: mds bal merge size = 5 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: mds bal split bits = 3 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: mds bal split size = 100 2024-05-09T08:55:12.230 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug frag = True 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds debug scatterstat = True 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds oft prefetch dirfrags = True 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds op complaint time = 180 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds verify scatter = True 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds_dir_prefetch = True 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mds_sleep_rank_change = 5000000.0 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: mon client directed command retry = 5 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay max = 1 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay probability = 0.005 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject delay type = client mds osd 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject socket failures = 2500 2024-05-09T08:55:12.231 INFO:teuthology.orchestra.run.smithi049.stdout: osd op complaint time = 180 2024-05-09T08:55:12.232 INFO:teuthology.orchestra.run.smithi049.stdout: rados mon op timeout = 900 2024-05-09T08:55:12.232 INFO:teuthology.orchestra.run.smithi049.stdout: rados osd op timeout = 900 2024-05-09T08:55:12.232 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T08:55:12.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T08:55:12.314 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T08:55:12.314 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:12.314 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T08:55:12.330 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:12.330 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T08:55:12.385 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:55:12.386 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:12.386 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T08:55:12.386 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:55:12.461 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:12.461 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:12.518 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:55:12.518 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:12.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T08:55:12.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T08:55:12.555 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:12.555 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:12.611 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T08:55:12.611 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T08:55:12.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T08:55:12.664 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T08:55:12.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T08:55:12.727 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T08:55:12.730 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T08:55:12.730 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T08:55:12.782 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T08:55:12.785 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T08:55:12.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T08:55:12.904 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T08:55:12.907 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T08:55:12.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-05-09T08:55:13.024 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-05-09T08:55:13.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-05-09T08:55:13.053 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T08:55:13.103 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T08:55:13.105 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T08:55:13.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T08:55:13.218 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T08:55:13.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T08:55:13.242 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-05-09T08:55:13.330 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-05-09T08:55:13.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-05-09T08:55:13.356 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T08:55:13.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T08:55:13.398 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T08:55:13.411 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T08:55:13.411 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T08:55:13.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T08:55:13.478 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:13.478 INFO:tasks.ceph:role: osd.0 2024-05-09T08:55:13.478 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:55:13.478 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T08:55:13.544 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:13.544 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:13.545 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:13.545 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:13.545 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:13.577 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-09T08:55:13.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-09T08:55:13.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T08:55:13.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T08:55:13.719 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T08:55:13.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:13.785 INFO:tasks.ceph:role: osd.1 2024-05-09T08:55:13.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:55:13.785 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T08:55:13.850 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:13.850 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:13.851 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:13.851 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:13.851 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:13.882 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-09T08:55:13.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-09T08:55:13.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T08:55:13.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T08:55:14.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T08:55:14.091 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:14.092 INFO:tasks.ceph:role: osd.2 2024-05-09T08:55:14.092 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:55:14.092 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:14.157 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:14.158 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:14.158 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:14.158 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:14.158 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:14.190 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-09T08:55:14.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-09T08:55:14.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T08:55:14.262 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T08:55:14.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T08:55:14.398 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:14.399 INFO:tasks.ceph:role: osd.3 2024-05-09T08:55:14.399 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:55:14.399 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:14.465 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:14.466 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:14.497 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-05-09T08:55:14.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-05-09T08:55:14.499 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T08:55:14.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:14.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T08:55:14.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T08:55:14.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:55:14.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:55:14.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T08:55:14.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:14.732+0000 7f9fc9e1a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T08:55:17.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T08:55:17.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.701+0000 7f210e1cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.701+0000 7f210e1cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.701+0000 7f210e1cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.701+0000 7f210e1cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.701+0000 7f210e1cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T08:55:17.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:17.702+0000 7f210e1cb640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T08:55:20.548 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T08:55:20.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.675+0000 7fefc6261640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.675+0000 7fefc6261640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.675+0000 7fefc6261640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.675+0000 7fefc6261640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.675+0000 7fefc6261640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T08:55:20.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:20.676+0000 7fefc6261640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T08:55:23.526 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T08:55:23.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.650+0000 7f94f0014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.650+0000 7f94f0014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.650+0000 7f94f0014640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.650+0000 7f94f0014640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.650+0000 7f94f0014640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T08:55:23.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:23.651+0000 7f94f0014640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T08:55:26.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T08:55:26.538 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T08:55:26.566 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:26.566 INFO:tasks.ceph:role: osd.4 2024-05-09T08:55:26.566 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:55:26.566 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:26.640 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:26.641 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:26.641 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:27.136 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-09T08:55:27.138 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-09T08:55:27.138 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T08:55:27.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T08:55:27.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T08:55:27.401 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:27.401 INFO:tasks.ceph:role: osd.5 2024-05-09T08:55:27.401 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:55:27.401 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:27.473 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:27.474 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:27.474 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:27.930 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-09T08:55:27.932 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-09T08:55:27.932 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T08:55:28.037 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T08:55:28.067 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T08:55:28.132 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:28.132 INFO:tasks.ceph:role: osd.6 2024-05-09T08:55:28.132 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:55:28.132 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:28.204 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:28.709 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-09T08:55:28.712 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-09T08:55:28.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T08:55:28.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T08:55:28.889 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T08:55:28.955 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T08:55:28.955 INFO:tasks.ceph:role: osd.7 2024-05-09T08:55:28.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:55:28.956 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T08:55:29.028 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T08:55:29.029 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T08:55:29.537 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-09T08:55:29.539 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-09T08:55:29.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T08:55:29.652 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T08:55:29.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:29.782 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.783+0000 7f558eadd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T08:55:29.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.783+0000 7f558eadd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T08:55:29.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.783+0000 7f558eadd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:55:29.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.783+0000 7f558eadd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:55:29.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.783+0000 7f558eadd640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T08:55:29.783 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:29.784+0000 7f558eadd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T08:55:32.631 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T08:55:32.659 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:32.746 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.747+0000 7f61e1921640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T08:55:32.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.747+0000 7f61e1921640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T08:55:32.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.747+0000 7f61e1921640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:55:32.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.747+0000 7f61e1921640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:55:32.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.747+0000 7f61e1921640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T08:55:32.747 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:32.748+0000 7f61e1921640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T08:55:35.607 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T08:55:35.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:35.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.722+0000 7f7464723640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T08:55:35.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.723+0000 7f7464723640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T08:55:35.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.723+0000 7f7464723640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:55:35.722 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.723+0000 7f7464723640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:55:35.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.723+0000 7f7464723640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T08:55:35.723 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:35.723+0000 7f7464723640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T08:55:38.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T08:55:38.612 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.700+0000 7f7fa4815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.700+0000 7f7fa4815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.700+0000 7f7fa4815640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.700+0000 7f7fa4815640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.700+0000 7f7fa4815640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T08:55:38.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:38.701+0000 7f7fa4815640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T08:55:40.237 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:40.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:55:41.554 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T08:55:41.582 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T08:55:41.582 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T08:55:41.614 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T08:55:41.684 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.684 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T08:55:41.753 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-05-09T08:55:41.822 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.822 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T08:55:41.890 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T08:55:41.959 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:41.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T08:55:42.026 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:42.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T08:55:42.092 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.092 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T08:55:42.121 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.121 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T08:55:42.187 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T08:55:42.254 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-05-09T08:55:42.320 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.320 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T08:55:42.386 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.387 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T08:55:42.453 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.453 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T08:55:42.518 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:42.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T08:55:42.582 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:42.582 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T08:55:42.600 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T08:55:42.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T08:55:42.642 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T08:55:42.665 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBwjzxm4B2zJxAAnb5P3pjvlIeFS4fdQM4+rQ== 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBwjzxmo9TGLhAAk7Fw06uiAyM3vKF2fD9jKQ== 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBwjzxmoE8FNhAAj7XUq4tjKBDS7b13BZHXbA== 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.e] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBxjzxm0C6YARAAnB53SJA48iJgoErEf9ppcw== 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQByjzxmsaWxKxAAdL1o7xHy1WkeDKZ8QPucmQ== 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2024-05-09T08:55:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQB1jzxmEIXfKRAA/S/fBcDoUBOC/2oKSD/e9Q== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQB4jzxm1FdSKBAAT8asCXPaWY8VtkX7KJDpzg== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQB7jzxm8THYJhAAnkXocXZEf699Z1pHd1qkUg== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBwjzxm1+F6KxAArP1sS86g6StFkdI3HKXCFg== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBxjzxmyd1CBhAAlTsE8XrLBzqOcfdGNPd8mg== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBxjzxmoroJDRAAKiGpQn36FYtfsCVImtPnrQ== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.f] 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBxjzxmn0nWExAA8OpIiCDbFgi65k9k666TPA== 2024-05-09T08:55:42.667 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCBjzxmguXALhAA6c9qLlnF3uOhxpCht6t0Zw== 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCEjzxmqVaYLBAAhmpMLMLQVcRVzkCrudpORg== 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCHjzxmFSAjKxAAICVsSjzmKDJrXzo0b4+AEA== 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQCKjzxm8CzQKRAAMifGBV+JntRNq8FSqt4DwA== 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2024-05-09T08:55:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBxjzxm5zbhFxAAeXiLUfhh7OP9fFpSd5Hf9A== 2024-05-09T08:55:42.669 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2024-05-09T08:55:42.669 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBwjzxm4B2zJxAAnb5P3pjvlIeFS4fdQM4+rQ== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.a] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBwjzxmo9TGLhAAk7Fw06uiAyM3vKF2fD9jKQ== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.c] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBwjzxmoE8FNhAAj7XUq4tjKBDS7b13BZHXbA== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.e] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBxjzxm0C6YARAAnB53SJA48iJgoErEf9ppcw== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQByjzxmsaWxKxAAdL1o7xHy1WkeDKZ8QPucmQ== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB1jzxmEIXfKRAA/S/fBcDoUBOC/2oKSD/e9Q== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB4jzxm1FdSKBAAT8asCXPaWY8VtkX7KJDpzg== 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2024-05-09T08:55:42.670 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQB7jzxm8THYJhAAnkXocXZEf699Z1pHd1qkUg== 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBwjzxm1+F6KxAArP1sS86g6StFkdI3HKXCFg== 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.b] 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBxjzxmyd1CBhAAlTsE8XrLBzqOcfdGNPd8mg== 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.d] 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBxjzxmoroJDRAAKiGpQn36FYtfsCVImtPnrQ== 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:[mds.f] 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBxjzxmn0nWExAA8OpIiCDbFgi65k9k666TPA== 2024-05-09T08:55:42.671 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCBjzxmguXALhAA6c9qLlnF3uOhxpCht6t0Zw== 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCEjzxmqVaYLBAAhmpMLMLQVcRVzkCrudpORg== 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCHjzxmFSAjKxAAICVsSjzmKDJrXzo0b4+AEA== 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQCKjzxm8CzQKRAAMifGBV+JntRNq8FSqt4DwA== 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-05-09T08:55:42.672 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBxjzxm5zbhFxAAeXiLUfhh7OP9fFpSd5Hf9A== 2024-05-09T08:55:42.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:55:42.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:55:42.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.767 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.877 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:42.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:42.932 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:42.982 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:42.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.037 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.086 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.088 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:55:43.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T08:55:43.191 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.249 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.304 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T08:55:43.356 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.460 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.514 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T08:55:43.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T08:55:43.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T08:55:43.619 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T08:55:43.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T08:55:43.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:55:43.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T08:55:43.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T08:55:43.791 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:55:43.886 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T08:55:43.909 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T08:55:43.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T08:55:44.066 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T08:55:44.089 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:44.091 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T08:55:44.143 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T08:55:44.144 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T08:55:44.144 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T08:55:44.147 INFO:tasks.ceph.mon.a:Started 2024-05-09T08:55:44.147 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T08:55:44.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T08:55:44.186 INFO:tasks.ceph.mon.b:Started 2024-05-09T08:55:44.186 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T08:55:44.186 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T08:55:44.188 INFO:tasks.ceph.mon.c:Started 2024-05-09T08:55:44.188 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T08:55:44.189 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T08:55:44.189 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T08:55:44.191 INFO:tasks.ceph.mgr.y:Started 2024-05-09T08:55:44.191 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T08:55:44.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T08:55:44.193 INFO:tasks.ceph.mgr.x:Started 2024-05-09T08:55:44.193 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T08:55:44.194 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T08:55:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 Processor -- start 2024-05-09T08:55:44.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 -- start start 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 0x7fcf300cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf300cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf300c6100 con 0x7fcf300ccb40 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fcf300b65c0 con 0x7fcf3015cbc0 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.298+0000 7fcf3669e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf30072a30 con 0x7fcf301632f0 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf27fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcf300ccb40 0x7fcf30162be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49848/0 (socket says 172.21.15.49:49848) 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf27fff640 1 -- 172.21.15.49:0/2496634919 learned_addr learned my addr 172.21.15.49:0/2496634919 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf2ffff640 1 --2- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 0x7fcf300cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf300cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf2ffff640 1 -- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 msgr2=0x7fcf300cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf2ffff640 1 -- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 msgr2=0x7fcf300cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf2ffff640 1 --2- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 0x7fcf300cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf2ffff640 1 --2- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 0x7fcf300cd260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 msgr2=0x7fcf300cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 msgr2=0x7fcf300cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf300cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:55:44.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.299+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf300cd7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.462+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842114925 0 0) 0x7fcf18001110 con 0x7fcf300ccb40 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.462+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf1c003520 con 0x7fcf300ccb40 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3641991183 0 0) 0x7fcf1c003520 con 0x7fcf300ccb40 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 msgr2=0x7fcf300cd7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf300cd7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 msgr2=0x7fcf300cd260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 --2- 172.21.15.49:0/2496634919 >> v2:172.21.15.183:3300/0 conn(0x7fcf3015cbc0 0x7fcf300cd260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf30072d10 con 0x7fcf300ccb40 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fcf18001840 con 0x7fcf300ccb40 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 >> v1:172.21.15.49:6789/0 conn(0x7fcf300ccb40 legacy=0x7fcf30162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:44.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1c003fe0 0x7fcf1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf1c004d40 0x7fcf1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf300b65c0 con 0x7fcf1c004d40 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30072a30 con 0x7fcf1c003fe0 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c00bac0 con 0x7fcf301632f0 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf27fff640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1c003fe0 0x7fcf1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf2ffff640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf1c004d40 0x7fcf1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 msgr2=0x7fcf1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf301632f0 0x7fcf1c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1c003fe0 msgr2=0x7fcf1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.463+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1c003fe0 0x7fcf1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.464+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c003520 con 0x7fcf1c004d40 2024-05-09T08:55:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.464+0000 7fcf34c3b640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf1c004d40 0x7fcf1c0051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009fd0 tx=0x7fcf1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cb9e0146b6dd75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.464+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28033070 con 0x7fcf1c004d40 2024-05-09T08:55:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.464+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2802f650 con 0x7fcf1c004d40 2024-05-09T08:55:44.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.464+0000 7fcf2f7fe640 1 -- 172.21.15.49:0/2496634919 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28009bd0 con 0x7fcf1c004d40 2024-05-09T08:55:44.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 -- 172.21.15.49:0/2496634919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf1c004d40 msgr2=0x7fcf1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 --2- 172.21.15.49:0/2496634919 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf1c004d40 0x7fcf1c0051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009fd0 tx=0x7fcf1c005830 comp rx=0 tx=0).stop 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf34c3b640 1 -- 172.21.15.49:0/2496634919 reap_dead start 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 -- 172.21.15.49:0/2496634919 shutdown_connections 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 -- 172.21.15.49:0/2496634919 >> 172.21.15.49:0/2496634919 conn(0x7fcf30074a20 msgr2=0x7fcf30074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 -- 172.21.15.49:0/2496634919 shutdown_connections 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.465+0000 7fcf3669e640 1 -- 172.21.15.49:0/2496634919 wait complete. 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 Processor -- start 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 -- start start 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 0x7fcf301f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 0x7fcf3015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf300c9330 0x7fcf301592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3015cfa0 con 0x7fcf300c9330 2024-05-09T08:55:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf300b6250 con 0x7fcf301623d0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.467+0000 7fcf3669e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf300c5800 con 0x7fcf1c004d40 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 0x7fcf301f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 0x7fcf3015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf34c3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf300c9330 0x7fcf301592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 0x7fcf301f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:37482/0 (socket says 172.21.15.49:37482) 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2ffff640 1 -- 172.21.15.49:0/2282146206 learned_addr learned my addr 172.21.15.49:0/2282146206 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 -- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 msgr2=0x7fcf301f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 0x7fcf301f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 -- 172.21.15.49:0/2282146206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf300c9330 msgr2=0x7fcf301592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf300c9330 0x7fcf301592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 -- 172.21.15.49:0/2282146206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf300c7e80 con 0x7fcf301623d0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf27fff640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 0x7fcf3015f670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007310 tx=0x7fcf180077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800f8e0 con 0x7fcf301623d0 2024-05-09T08:55:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18018070 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf300c8160 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.468+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800fcc0 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.469+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301f8690 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.469+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf1801d460 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.469+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf18013700 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.469+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3004f660 con 0x7fcf301623d0 2024-05-09T08:55:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.472+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcf300c6100 con 0x7fcf301623d0 2024-05-09T08:55:44.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.538+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcf300c5b30 con 0x7fcf301623d0 2024-05-09T08:55:44.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.544+0000 7fcf2d7fa640 1 -- 172.21.15.49:0/2282146206 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fcf1800fe60 con 0x7fcf301623d0 2024-05-09T08:55:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 msgr2=0x7fcf3015f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 0x7fcf3015f670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007310 tx=0x7fcf180077d0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 shutdown_connections 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf300c9330 0x7fcf301592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf301623d0 0x7fcf3015f670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 --2- 172.21.15.49:0/2282146206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf1c004d40 0x7fcf301f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 >> 172.21.15.49:0/2282146206 conn(0x7fcf30074a20 msgr2=0x7fcf3015ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 shutdown_connections 2024-05-09T08:55:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:44.545+0000 7fcf3669e640 1 -- 172.21.15.49:0/2282146206 wait complete. 2024-05-09T08:55:44.555 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T08:55:44.555 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T08:55:44.556 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:44.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T08:55:44.569 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:44.569+0000 7f0c729e6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:55:44.580 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:44.581+0000 7fea81529040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T08:55:44.582 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:44.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T08:55:44.648 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:44.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T08:55:44.715 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:55:44.716 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T08:55:44.719 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:44.719+0000 7f0c729e6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:55:44.748 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:44.749+0000 7fea81529040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:55:44.771 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:44.771+0000 7f0c729e6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:55:44.773 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:44.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T08:55:44.797 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:44.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T08:55:44.806 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:44.807+0000 7fea81529040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T08:55:44.860 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:44.860 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T08:55:44.923 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-09T08:55:44.923 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T08:55:44.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new b59d26bf-6806-4e49-b117-9d83d3a24243 0 2024-05-09T08:55:45.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:45.016+0000 7f0c729e6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:55:45.061 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:45.062+0000 7fea81529040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T08:55:45.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:45.065+0000 7f0c729e6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:55:45.109 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:45.110+0000 7fea81529040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T08:55:45.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.111+0000 7f0bc1fee640 1 Processor -- start 2024-05-09T08:55:45.112 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.111+0000 7f0bc1fee640 1 -- start start 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bc1fee640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0bbc0c9500 0x7f0bbc0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bc1fee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bc1fee640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bbc070990 con 0x7f0bbc152f30 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bc1fee640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0bbc0c5fa0 con 0x7f0bbc0c9500 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bc1fee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc0ca5f0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bbbfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bbb7fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0bbc152f30 0x7f0bbc0c5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56504/0 (socket says 172.21.15.183:56504) 2024-05-09T08:55:45.114 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bbb7fe640 1 -- 172.21.15.183:0/3059391418 learned_addr learned my addr 172.21.15.183:0/3059391418 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.112+0000 7f0bbaffd640 1 --2- 172.21.15.183:0/3059391418 >> v2:172.21.15.49:3300/0 conn(0x7f0bbc0c9500 0x7f0bbc0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bba7fc640 1 -- 172.21.15.183:0/3059391418 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629398598 0 0) 0x7f0bbc070990 con 0x7f0bbc152f30 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bba7fc640 1 -- 172.21.15.183:0/3059391418 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ba4003520 con 0x7f0bbc152f30 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bbbfff640 1 -- 172.21.15.183:0/3059391418 >> v2:172.21.15.49:3300/0 conn(0x7f0bbc0c9500 msgr2=0x7f0bbc0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bbbfff640 1 --2- 172.21.15.183:0/3059391418 >> v2:172.21.15.49:3300/0 conn(0x7f0bbc0c9500 0x7f0bbc0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bbbfff640 1 -- 172.21.15.183:0/3059391418 >> v1:172.21.15.183:6789/0 conn(0x7f0bbc152f30 legacy=0x7f0bbc0c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bbbfff640 1 -- 172.21.15.183:0/3059391418 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc15ccd0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.113+0000 7f0bbbfff640 1 --2- 172.21.15.183:0/3059391418 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0c7b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009d90 tx=0x7f0bb402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd911878abb2cc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bba7fc640 1 -- 172.21.15.183:0/3059391418 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4007740 con 0x7f0bbc0733c0 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bba7fc640 1 -- 172.21.15.183:0/3059391418 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb40078e0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bba7fc640 1 -- 172.21.15.183:0/3059391418 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4007cb0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 -- 172.21.15.183:0/3059391418 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 msgr2=0x7f0bbc0c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/3059391418 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0c7b40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009d90 tx=0x7f0bb402a6f0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 -- 172.21.15.183:0/3059391418 shutdown_connections 2024-05-09T08:55:45.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/3059391418 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0c7b40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/3059391418 >> v2:172.21.15.49:3300/0 conn(0x7f0bbc0c9500 0x7f0bbc0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 -- 172.21.15.183:0/3059391418 >> 172.21.15.183:0/3059391418 conn(0x7f0bbc0ce390 msgr2=0x7f0bbc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 -- 172.21.15.183:0/3059391418 shutdown_connections 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.114+0000 7f0bc1fee640 1 -- 172.21.15.183:0/3059391418 wait complete. 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.115+0000 7f0bc1fee640 1 Processor -- start 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 -- start start 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bbc0c9500 0x7f0bbc06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc152f30 0x7f0bbc06bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc0caa80 con 0x7f0bbc0c9500 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc0ca1a0 con 0x7f0bbc152f30 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bc1fee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc14e9b0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.116+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34404/0 (socket says 172.21.15.183:34404) 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 -- 172.21.15.183:0/2914110929 learned_addr learned my addr 172.21.15.183:0/2914110929 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbbfff640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc152f30 0x7f0bbc06bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbaffd640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bbc0c9500 0x7f0bbc06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 -- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc152f30 msgr2=0x7f0bbc06bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc152f30 0x7f0bbc06bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 -- 172.21.15.183:0/2914110929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bbc0c9500 msgr2=0x7f0bbc06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bbc0c9500 0x7f0bbc06cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 -- 172.21.15.183:0/2914110929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc14f250 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bbb7fe640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c3c0 tx=0x7f0bb000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0017070 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb0005ce0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc14f510 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00072f0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc14d270 con 0x7f0bbc0733c0 2024-05-09T08:55:45.121 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0bb001a070 con 0x7f0bbc0733c0 2024-05-09T08:55:45.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.117+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f0bb0012740 con 0x7f0bbc0733c0 2024-05-09T08:55:45.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.118+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc0c9fc0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.120+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0bbc0c9fc0 con 0x7f0bbc0733c0 2024-05-09T08:55:45.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.192+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b59d26bf-6806-4e49-b117-9d83d3a24243", "id": 0} v 0) -- 0x7f0bbc04f660 con 0x7f0bbc0733c0 2024-05-09T08:55:45.196 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.196+0000 7f0ba3fff640 1 -- 172.21.15.183:0/2914110929 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b59d26bf-6806-4e49-b117-9d83d3a24243", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f0bb0021020 con 0x7f0bbc0733c0 2024-05-09T08:55:45.196 INFO:teuthology.orchestra.run.smithi183.stdout:0 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.196+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 msgr2=0x7f0bbc0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.196+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c3c0 tx=0x7f0bb000c880 comp rx=0 tx=0).stop 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 shutdown_connections 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bbc152f30 0x7f0bbc06bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bbc0c9500 0x7f0bbc06cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 --2- 172.21.15.183:0/2914110929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bbc0733c0 0x7f0bbc0728a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 >> 172.21.15.183:0/2914110929 conn(0x7f0bbc0ce390 msgr2=0x7f0bbc150c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 shutdown_connections 2024-05-09T08:55:45.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.197+0000 7f0bc1fee640 1 -- 172.21.15.183:0/2914110929 wait complete. 2024-05-09T08:55:45.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new d579c6a2-0370-4c21-96fb-6420583dbe74 1 2024-05-09T08:55:45.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.303+0000 7f3e5effb640 1 Processor -- start 2024-05-09T08:55:45.303 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.303+0000 7f3e5effb640 1 -- start start 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5effb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3e58074ab0 0x7f3e580c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5effb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e580c9570 con 0x7f3e58074e90 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5effb640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e580c9850 con 0x7f3e58073330 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5effb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3e580c8e00 con 0x7f3e58074ab0 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5cd97640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3e58074ab0 0x7f3e580c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5d598640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3e58073330 0x7f3e58154ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56512/0 (socket says 172.21.15.183:56512) 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5d598640 1 -- 172.21.15.183:0/3678701666 learned_addr learned my addr 172.21.15.183:0/3678701666 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5cd97640 1 -- 172.21.15.183:0/3678701666 >> v1:172.21.15.183:6789/0 conn(0x7f3e58073330 legacy=0x7f3e58154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5cd97640 1 -- 172.21.15.183:0/3678701666 >> v1:172.21.15.49:6789/0 conn(0x7f3e58074e90 legacy=0x7f3e5815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.304+0000 7f3e5cd97640 1 -- 172.21.15.183:0/3678701666 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580711e0 con 0x7f3e58074ab0 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.305+0000 7f3e5cd97640 1 --2- 172.21.15.183:0/3678701666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074ab0 0x7f3e580c88c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0097a0 tx=0x7f3e4c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e883bf6ea5aa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.305+0000 7f3e577fe640 1 -- 172.21.15.183:0/3678701666 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c037070 con 0x7f3e58074ab0 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.305+0000 7f3e577fe640 1 -- 172.21.15.183:0/3678701666 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c032440 con 0x7f3e58074ab0 2024-05-09T08:55:45.306 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.305+0000 7f3e577fe640 1 -- 172.21.15.183:0/3678701666 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c032810 con 0x7f3e58074ab0 2024-05-09T08:55:45.307 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 -- 172.21.15.183:0/3678701666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074ab0 msgr2=0x7f3e580c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.307 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 --2- 172.21.15.183:0/3678701666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074ab0 0x7f3e580c88c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c0097a0 tx=0x7f3e4c02a230 comp rx=0 tx=0).stop 2024-05-09T08:55:45.307 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 -- 172.21.15.183:0/3678701666 shutdown_connections 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 --2- 172.21.15.183:0/3678701666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074ab0 0x7f3e580c88c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 -- 172.21.15.183:0/3678701666 >> 172.21.15.183:0/3678701666 conn(0x7f3e5806b200 msgr2=0x7f3e5806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 -- 172.21.15.183:0/3678701666 shutdown_connections 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.306+0000 7f3e5effb640 1 -- 172.21.15.183:0/3678701666 wait complete. 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.307+0000 7f3e5effb640 1 Processor -- start 2024-05-09T08:55:45.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.307+0000 7f3e5effb640 1 -- start start 2024-05-09T08:55:45.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e58073330 0x7f3e580c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e58074ab0 0x7f3e580c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.310 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580c9b30 con 0x7f3e58073330 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e580717a0 con 0x7f3e58074e90 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3e58071cc0 con 0x7f3e58074ab0 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57622/0 (socket says 172.21.15.183:57622) 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 -- 172.21.15.183:0/2857268971 learned_addr learned my addr 172.21.15.183:0/2857268971 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e57fff640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e58074ab0 0x7f3e580c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 -- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e58074ab0 msgr2=0x7f3e580c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e58074ab0 0x7f3e580c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 -- 172.21.15.183:0/2857268971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e58073330 msgr2=0x7f3e580c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e58073330 0x7f3e580c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 -- 172.21.15.183:0/2857268971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e580c5720 con 0x7f3e58074e90 2024-05-09T08:55:45.311 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5d598640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000cae0 tx=0x7f3e5000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50017070 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5000ac40 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50012400 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e580c5170 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.308+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e580cb570 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.309+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e5001a070 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.309+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3e5001c870 con 0x7f3e58074e90 2024-05-09T08:55:45.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.309+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5804f660 con 0x7f3e58074e90 2024-05-09T08:55:45.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.314+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3e5001ca70 con 0x7f3e58074e90 2024-05-09T08:55:45.383 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.383+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d579c6a2-0370-4c21-96fb-6420583dbe74", "id": 1} v 0) -- 0x7f3e58145e40 con 0x7f3e58074e90 2024-05-09T08:55:45.389 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.389+0000 7f3e557fa640 1 -- 172.21.15.183:0/2857268971 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d579c6a2-0370-4c21-96fb-6420583dbe74", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3e5001cc50 con 0x7f3e58074e90 2024-05-09T08:55:45.389 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-05-09T08:55:45.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 msgr2=0x7f3e580c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000cae0 tx=0x7f3e5000a060 comp rx=0 tx=0).stop 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 shutdown_connections 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e58074e90 0x7f3e580c6010 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e58074ab0 0x7f3e580c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 --2- 172.21.15.183:0/2857268971 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e58073330 0x7f3e580c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 >> 172.21.15.183:0/2857268971 conn(0x7f3e5806b200 msgr2=0x7f3e5806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 shutdown_connections 2024-05-09T08:55:45.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.390+0000 7f3e5effb640 1 -- 172.21.15.183:0/2857268971 wait complete. 2024-05-09T08:55:45.400 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new bb2ad0e5-2ec1-4647-bc42-3d20911e038a 2 2024-05-09T08:55:45.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.497+0000 7f690c815640 1 Processor -- start 2024-05-09T08:55:45.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.497+0000 7f690c815640 1 -- start start 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690c815640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69080cd280 0x7f69080c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690c815640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69080cdc50 con 0x7f69080cd710 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690c815640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6908157ac0 con 0x7f6908151c00 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690c815640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6908157da0 con 0x7f69080cd280 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f6905d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69080cd280 0x7f69080c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690659c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6908151c00 0x7f69080c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56518/0 (socket says 172.21.15.183:56518) 2024-05-09T08:55:45.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.498+0000 7f690659c640 1 -- 172.21.15.183:0/721137572 learned_addr learned my addr 172.21.15.183:0/721137572 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:45.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.499+0000 7f6905d9b640 1 -- 172.21.15.183:0/721137572 >> v1:172.21.15.183:6789/0 conn(0x7f6908151c00 legacy=0x7f69080c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.499+0000 7f6905d9b640 1 -- 172.21.15.183:0/721137572 >> v1:172.21.15.49:6789/0 conn(0x7f69080cd710 legacy=0x7f690815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.499+0000 7f6905d9b640 1 -- 172.21.15.183:0/721137572 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6908158080 con 0x7f69080cd280 2024-05-09T08:55:45.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.499+0000 7f690559a640 1 -- 172.21.15.183:0/721137572 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130271283 0 0) 0x7f6908157ac0 con 0x7f6908151c00 2024-05-09T08:55:45.501 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.500+0000 7f6905d9b640 1 --2- 172.21.15.183:0/721137572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd280 0x7f69080c8180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0097a0 tx=0x7f68fc02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e66cc1ec442012 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.501 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.500+0000 7f690559a640 1 -- 172.21.15.183:0/721137572 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc037070 con 0x7f69080cd280 2024-05-09T08:55:45.501 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.500+0000 7f690559a640 1 -- 172.21.15.183:0/721137572 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc032640 con 0x7f69080cd280 2024-05-09T08:55:45.501 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.500+0000 7f690559a640 1 -- 172.21.15.183:0/721137572 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc032a10 con 0x7f69080cd280 2024-05-09T08:55:45.501 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 -- 172.21.15.183:0/721137572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd280 msgr2=0x7f69080c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 --2- 172.21.15.183:0/721137572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd280 0x7f69080c8180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68fc0097a0 tx=0x7f68fc02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 -- 172.21.15.183:0/721137572 shutdown_connections 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 --2- 172.21.15.183:0/721137572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd280 0x7f69080c8180 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 -- 172.21.15.183:0/721137572 >> 172.21.15.183:0/721137572 conn(0x7f690806b200 msgr2=0x7f690806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 -- 172.21.15.183:0/721137572 shutdown_connections 2024-05-09T08:55:45.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.501+0000 7f690c815640 1 -- 172.21.15.183:0/721137572 wait complete. 2024-05-09T08:55:45.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.502+0000 7f690c815640 1 Processor -- start 2024-05-09T08:55:45.503 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.502+0000 7f690c815640 1 -- start start 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 0x7f690814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908151c00 0x7f6908147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6908151fe0 con 0x7f6908151c00 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6908158640 con 0x7f69080cd710 2024-05-09T08:55:45.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690c815640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f690815b7b0 con 0x7f69080cd280 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690659c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 0x7f690814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690659c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 0x7f690814cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34414/0 (socket says 172.21.15.183:34414) 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57632/0 (socket says 172.21.15.183:57632) 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f690659c640 1 -- 172.21.15.183:0/4089607822 learned_addr learned my addr 172.21.15.183:0/4089607822 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6906d9d640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908151c00 0x7f6908147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 -- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 msgr2=0x7f690814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 0x7f690814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 -- 172.21.15.183:0/4089607822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908151c00 msgr2=0x7f6908147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908151c00 0x7f6908147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.503+0000 7f6905d9b640 1 -- 172.21.15.183:0/4089607822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69080cbf90 con 0x7f69080cd710 2024-05-09T08:55:45.506 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f6905d9b640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68fc031000 tx=0x7f68fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc037040 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc042070 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc030bb0 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690814c540 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69080cbcb0 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f68fc047460 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.504+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f68fc032510 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.505+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690804f660 con 0x7f69080cd710 2024-05-09T08:55:45.507 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.507+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f68fc051480 con 0x7f69080cd710 2024-05-09T08:55:45.579 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.579+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bb2ad0e5-2ec1-4647-bc42-3d20911e038a", "id": 2} v 0) -- 0x7f690806b950 con 0x7f69080cd710 2024-05-09T08:55:45.582 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.582+0000 7f68f6ffd640 1 -- 172.21.15.183:0/4089607822 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb2ad0e5-2ec1-4647-bc42-3d20911e038a", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f68fc051760 con 0x7f69080cd710 2024-05-09T08:55:45.583 INFO:teuthology.orchestra.run.smithi183.stdout:2 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 msgr2=0x7f69080c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f68fc031000 tx=0x7f68fc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 shutdown_connections 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6908151c00 0x7f6908147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69080cd710 0x7f69080c6c50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 --2- 172.21.15.183:0/4089607822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69080cd280 0x7f690814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 >> 172.21.15.183:0/4089607822 conn(0x7f690806b200 msgr2=0x7f69080c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 shutdown_connections 2024-05-09T08:55:45.584 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.583+0000 7f690c815640 1 -- 172.21.15.183:0/4089607822 wait complete. 2024-05-09T08:55:45.594 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 546a2935-7144-4c29-b647-e77d89072d60 3 2024-05-09T08:55:45.693 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 Processor -- start 2024-05-09T08:55:45.693 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 -- start start 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0bf806f580 0x7f0bf806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 0x7f0bf815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bf8155460 con 0x7f0bf8155080 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0bf815ba00 con 0x7f0bf806f580 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.693+0000 7f0bff01b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf806fae0 con 0x7f0bf8073310 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfd5b8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 0x7f0bf815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.695 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfcdb7640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0bf8155080 0x7f0bf8074840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56532/0 (socket says 172.21.15.183:56532) 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfcdb7640 1 -- 172.21.15.183:0/1506759278 learned_addr learned my addr 172.21.15.183:0/1506759278 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bf7fff640 1 --2- 172.21.15.183:0/1506759278 >> v2:172.21.15.49:3300/0 conn(0x7f0bf806f580 0x7f0bf806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bf77fe640 1 -- 172.21.15.183:0/1506759278 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1771793509 0 0) 0x7f0bf8155460 con 0x7f0bf8155080 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/1506759278 >> v2:172.21.15.49:3300/0 conn(0x7f0bf806f580 msgr2=0x7f0bf806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfd5b8640 1 --2- 172.21.15.183:0/1506759278 >> v2:172.21.15.49:3300/0 conn(0x7f0bf806f580 0x7f0bf806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/1506759278 >> v1:172.21.15.183:6789/0 conn(0x7f0bf8155080 legacy=0x7f0bf8074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.694+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/1506759278 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf815adf0 con 0x7f0bf8073310 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bfd5b8640 1 --2- 172.21.15.183:0/1506759278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 0x7f0bf815e340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009dc0 tx=0x7f0bf002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaad280d718f38aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bf77fe640 1 -- 172.21.15.183:0/1506759278 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007740 con 0x7f0bf8073310 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bf77fe640 1 -- 172.21.15.183:0/1506759278 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf00078e0 con 0x7f0bf8073310 2024-05-09T08:55:45.696 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bf77fe640 1 -- 172.21.15.183:0/1506759278 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0007cb0 con 0x7f0bf8073310 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 -- 172.21.15.183:0/1506759278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 msgr2=0x7f0bf815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 --2- 172.21.15.183:0/1506759278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 0x7f0bf815e340 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009dc0 tx=0x7f0bf002a710 comp rx=0 tx=0).stop 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 -- 172.21.15.183:0/1506759278 shutdown_connections 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 --2- 172.21.15.183:0/1506759278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8073310 0x7f0bf815e340 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 --2- 172.21.15.183:0/1506759278 >> v2:172.21.15.49:3300/0 conn(0x7f0bf806f580 0x7f0bf806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 -- 172.21.15.183:0/1506759278 >> 172.21.15.183:0/1506759278 conn(0x7f0bf806b200 msgr2=0x7f0bf806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 -- 172.21.15.183:0/1506759278 shutdown_connections 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.695+0000 7f0bff01b640 1 -- 172.21.15.183:0/1506759278 wait complete. 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 Processor -- start 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 -- start start 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 0x7f0bf815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.700 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bf8073310 0x7f0bf81f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 0x7f0bf8142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf815b690 con 0x7f0bf8073310 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf815b3b0 con 0x7f0bf806f580 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.697+0000 7f0bff01b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf8074f50 con 0x7f0bf8155080 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfcdb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 0x7f0bf815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 0x7f0bf8142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfcdb7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 0x7f0bf815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57638/0 (socket says 172.21.15.183:57638) 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfcdb7640 1 -- 172.21.15.183:0/3080153813 learned_addr learned my addr 172.21.15.183:0/3080153813 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf7fff640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bf8073310 0x7f0bf81f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 msgr2=0x7f0bf815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 0x7f0bf815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/3080153813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bf8073310 msgr2=0x7f0bf81f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.701 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bf8073310 0x7f0bf81f1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 -- 172.21.15.183:0/3080153813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf81468a0 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bfd5b8640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 0x7f0bf8142d10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0031000 tx=0x7f0bf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0040070 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf0031b40 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf815c250 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0045460 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf815c810 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0bf0039040 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.698+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f0bf0045d30 con 0x7f0bf8155080 2024-05-09T08:55:45.702 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.699+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf8154c30 con 0x7f0bf8155080 2024-05-09T08:55:45.703 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.701+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0bf8154c30 con 0x7f0bf8155080 2024-05-09T08:55:45.772 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.772+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "546a2935-7144-4c29-b647-e77d89072d60", "id": 3} v 0) -- 0x7f0bf804f660 con 0x7f0bf8155080 2024-05-09T08:55:45.776 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.776+0000 7f0bf57fa640 1 -- 172.21.15.183:0/3080153813 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "546a2935-7144-4c29-b647-e77d89072d60", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f0bf0053080 con 0x7f0bf8155080 2024-05-09T08:55:45.776 INFO:teuthology.orchestra.run.smithi183.stdout:3 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 msgr2=0x7f0bf8142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 0x7f0bf8142d10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0031000 tx=0x7f0bf0004a40 comp rx=0 tx=0).stop 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 shutdown_connections 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0bf8155080 0x7f0bf8142d10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0bf8073310 0x7f0bf81f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 --2- 172.21.15.183:0/3080153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bf806f580 0x7f0bf815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.777 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 >> 172.21.15.183:0/3080153813 conn(0x7f0bf806b200 msgr2=0x7f0bf80c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.778 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 shutdown_connections 2024-05-09T08:55:45.778 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.777+0000 7f0bff01b640 1 -- 172.21.15.183:0/3080153813 wait complete. 2024-05-09T08:55:45.788 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new f4bb828b-6036-4886-aec7-be53c00ac603 4 2024-05-09T08:55:45.829 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:45.829+0000 7f0c729e6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:55:45.873 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:45.874+0000 7fea81529040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T08:55:45.884 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.884+0000 7f53595f9640 1 Processor -- start 2024-05-09T08:55:45.884 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.884+0000 7f53595f9640 1 -- start start 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53595f9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f53540c6d50 0x7f5354071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53595f9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5354072600 con 0x7f5354154b90 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53595f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53540728e0 con 0x7f5354154f70 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53595f9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f53540c5880 con 0x7f53540c6d50 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53527fc640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f5354154f70 0x7f53540c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56546/0 (socket says 172.21.15.183:56546) 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53527fc640 1 -- 172.21.15.183:0/3211511686 learned_addr learned my addr 172.21.15.183:0/3211511686 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:45.886 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53537fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f53540c6d50 0x7f5354071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53537fe640 1 -- 172.21.15.183:0/3211511686 >> v1:172.21.15.183:6789/0 conn(0x7f5354154f70 legacy=0x7f53540c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53537fe640 1 -- 172.21.15.183:0/3211511686 >> v1:172.21.15.49:6789/0 conn(0x7f5354154b90 legacy=0x7f53540cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f5351ffb640 1 -- 172.21.15.183:0/3211511686 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449998899 0 0) 0x7f53540728e0 con 0x7f5354154f70 2024-05-09T08:55:45.887 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.885+0000 7f53537fe640 1 -- 172.21.15.183:0/3211511686 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53540c5b60 con 0x7f53540c6d50 2024-05-09T08:55:45.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.886+0000 7f53537fe640 1 --2- 172.21.15.183:0/3211511686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53540c6d50 0x7f5354071820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f534c009d90 tx=0x7f534c02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba7b8a671f943b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.886+0000 7f5351ffb640 1 -- 172.21.15.183:0/3211511686 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c037070 con 0x7f53540c6d50 2024-05-09T08:55:45.888 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.886+0000 7f5351ffb640 1 -- 172.21.15.183:0/3211511686 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c007740 con 0x7f53540c6d50 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.886+0000 7f5351ffb640 1 -- 172.21.15.183:0/3211511686 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c007b10 con 0x7f53540c6d50 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 -- 172.21.15.183:0/3211511686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53540c6d50 msgr2=0x7f5354071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 --2- 172.21.15.183:0/3211511686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53540c6d50 0x7f5354071820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f534c009d90 tx=0x7f534c02a780 comp rx=0 tx=0).stop 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 -- 172.21.15.183:0/3211511686 shutdown_connections 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 --2- 172.21.15.183:0/3211511686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53540c6d50 0x7f5354071820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 -- 172.21.15.183:0/3211511686 >> 172.21.15.183:0/3211511686 conn(0x7f5354074840 msgr2=0x7f5354074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 -- 172.21.15.183:0/3211511686 shutdown_connections 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.887+0000 7f53595f9640 1 -- 172.21.15.183:0/3211511686 wait complete. 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.888+0000 7f53595f9640 1 Processor -- start 2024-05-09T08:55:45.889 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.888+0000 7f53595f9640 1 -- start start 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 0x7f5354157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5354154f70 0x7f535414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f535415af40 con 0x7f5354154f70 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53540c6120 con 0x7f5354154b90 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53595f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53540cb990 con 0x7f53540c6d50 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.892 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f5352ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 0x7f5354157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57648/0 (socket says 172.21.15.183:57648) 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 -- 172.21.15.183:0/3165199780 learned_addr learned my addr 172.21.15.183:0/3165199780 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53537fe640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5354154f70 0x7f535414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 -- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 msgr2=0x7f5354157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 0x7f5354157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 -- 172.21.15.183:0/3165199780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5354154f70 msgr2=0x7f535414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5354154f70 0x7f535414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f53527fc640 1 -- 172.21.15.183:0/3165199780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354156580 con 0x7f5354154b90 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.889+0000 7f5352ffd640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 0x7f5354157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f53527fc640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f534800c3f0 tx=0x7f534800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53480074e0 con 0x7f5354154b90 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53480047e0 con 0x7f5354154b90 2024-05-09T08:55:45.893 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348004c10 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5354156860 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535414ee90 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5348011070 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.890+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f53480138d0 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.891+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5354156d80 con 0x7f5354154b90 2024-05-09T08:55:45.894 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.893+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5354156d80 con 0x7f5354154b90 2024-05-09T08:55:45.921 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:45.920+0000 7f0c729e6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:55:45.963 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:45.963+0000 7f0c729e6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:55:45.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.963+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f4bb828b-6036-4886-aec7-be53c00ac603", "id": 4} v 0) -- 0x7f53540d14e0 con 0x7f5354154b90 2024-05-09T08:55:45.964 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:45.965+0000 7fea81529040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T08:55:45.967 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f533b7fe640 1 -- 172.21.15.183:0/3165199780 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4bb828b-6036-4886-aec7-be53c00ac603", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f5348013b80 con 0x7f5354154b90 2024-05-09T08:55:45.967 INFO:teuthology.orchestra.run.smithi183.stdout:4 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 msgr2=0x7f535414c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f534800c3f0 tx=0x7f534800c8b0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 shutdown_connections 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5354154f70 0x7f535414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5354154b90 0x7f535414c9c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 --2- 172.21.15.183:0/3165199780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53540c6d50 0x7f5354157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:45.968 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.967+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 >> 172.21.15.183:0/3165199780 conn(0x7f5354074840 msgr2=0x7f5354071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:45.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.968+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 shutdown_connections 2024-05-09T08:55:45.969 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:45.968+0000 7f53595f9640 1 -- 172.21.15.183:0/3165199780 wait complete. 2024-05-09T08:55:45.978 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 41625c33-9f16-4279-b2aa-a63ac0aa6320 5 2024-05-09T08:55:46.008 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.009+0000 7fea81529040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T08:55:46.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.115+0000 7fdb460df640 1 Processor -- start 2024-05-09T08:55:46.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.115+0000 7fdb460df640 1 -- start start 2024-05-09T08:55:46.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.115+0000 7fdb460df640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 0x7fdb400c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb460df640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb460df640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb40075330 con 0x7fdb40152ff0 2024-05-09T08:55:46.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb460df640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdb40074840 con 0x7fdb400c8630 2024-05-09T08:55:46.116 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb460df640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdb401533d0 con 0x7fdb400c8aa0 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 0x7fdb400c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3effd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 0x7fdb400c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57660/0 (socket says 172.21.15.183:57660) 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34460/0 (socket says 172.21.15.183:34460) 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3f7fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fdb40152ff0 0x7fdb4006ed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56556/0 (socket says 172.21.15.183:56556) 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3effd640 1 -- 172.21.15.183:0/1647058592 learned_addr learned my addr 172.21.15.183:0/1647058592 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1647058592 >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 msgr2=0x7fdb400c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 --2- 172.21.15.183:0/1647058592 >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 0x7fdb400c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1647058592 >> v1:172.21.15.183:6789/0 conn(0x7fdb40152ff0 legacy=0x7fdb4006ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.116+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1647058592 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb400c8f50 con 0x7fdb400c8aa0 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb3ffff640 1 --2- 172.21.15.183:0/1647058592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009d90 tx=0x7fdb3802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de438de0d12bc8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb3e7fc640 1 -- 172.21.15.183:0/1647058592 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007740 con 0x7fdb400c8aa0 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb3e7fc640 1 -- 172.21.15.183:0/1647058592 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb380078e0 con 0x7fdb400c8aa0 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb3e7fc640 1 -- 172.21.15.183:0/1647058592 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007cb0 con 0x7fdb400c8aa0 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 -- 172.21.15.183:0/1647058592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 msgr2=0x7fdb400c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.119 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 --2- 172.21.15.183:0/1647058592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009d90 tx=0x7fdb3802a660 comp rx=0 tx=0).stop 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 -- 172.21.15.183:0/1647058592 shutdown_connections 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 --2- 172.21.15.183:0/1647058592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb400c8aa0 0x7fdb400c9770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 --2- 172.21.15.183:0/1647058592 >> v2:172.21.15.183:3300/0 conn(0x7fdb400c8630 0x7fdb400c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.117+0000 7fdb460df640 1 -- 172.21.15.183:0/1647058592 >> 172.21.15.183:0/1647058592 conn(0x7fdb4006b200 msgr2=0x7fdb4006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.118+0000 7fdb460df640 1 -- 172.21.15.183:0/1647058592 shutdown_connections 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.118+0000 7fdb460df640 1 -- 172.21.15.183:0/1647058592 wait complete. 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.119+0000 7fdb460df640 1 Processor -- start 2024-05-09T08:55:46.120 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.119+0000 7fdb460df640 1 -- start start 2024-05-09T08:55:46.122 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.119+0000 7fdb460df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb400c8630 0x7fdb40157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb400c8aa0 0x7fdb40143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40075050 con 0x7fdb400c8630 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40074d70 con 0x7fdb400c8aa0 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdb40158fe0 con 0x7fdb40152ff0 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb400c8aa0 0x7fdb40143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34470/0 (socket says 172.21.15.183:34470) 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1081866340 learned_addr learned my addr 172.21.15.183:0/1081866340 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3f7fe640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb400c8630 0x7fdb40157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb400c8aa0 msgr2=0x7fdb40143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb400c8aa0 0x7fdb40143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.123 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1081866340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb400c8630 msgr2=0x7fdb40157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb400c8630 0x7fdb40157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 -- 172.21.15.183:0/1081866340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb401ef170 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb3ffff640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb3802ac50 tx=0x7fdb38004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38040070 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb38031bc0 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb401ef450 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38009450 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.120+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb40143620 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.121+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb38039040 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.121+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fdb38045df0 con 0x7fdb40152ff0 2024-05-09T08:55:46.124 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.121+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb40143820 con 0x7fdb40152ff0 2024-05-09T08:55:46.125 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.123+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdb40143820 con 0x7fdb40152ff0 2024-05-09T08:55:46.136 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:46.136+0000 7f0c729e6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:55:46.182 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.183+0000 7fea81529040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T08:55:46.186 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:46.187+0000 7f0c729e6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:55:46.195 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.196+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "41625c33-9f16-4279-b2aa-a63ac0aa6320", "id": 5} v 0) -- 0x7fdb4004f660 con 0x7fdb40152ff0 2024-05-09T08:55:46.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.199+0000 7fdb27fff640 1 -- 172.21.15.183:0/1081866340 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "41625c33-9f16-4279-b2aa-a63ac0aa6320", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fdb38007840 con 0x7fdb40152ff0 2024-05-09T08:55:46.198 INFO:teuthology.orchestra.run.smithi183.stdout:5 2024-05-09T08:55:46.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 msgr2=0x7fdb40144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb3802ac50 tx=0x7fdb38004a40 comp rx=0 tx=0).stop 2024-05-09T08:55:46.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 shutdown_connections 2024-05-09T08:55:46.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb40152ff0 0x7fdb40144080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb400c8aa0 0x7fdb40143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.200 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 --2- 172.21.15.183:0/1081866340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb400c8630 0x7fdb40157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.200 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 >> 172.21.15.183:0/1081866340 conn(0x7fdb4006b200 msgr2=0x7fdb400c6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.200 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 shutdown_connections 2024-05-09T08:55:46.200 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.200+0000 7fdb460df640 1 -- 172.21.15.183:0/1081866340 wait complete. 2024-05-09T08:55:46.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 9177552d-bc69-409e-9654-e030c3a1afba 6 2024-05-09T08:55:46.232 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.233+0000 7fea81529040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T08:55:46.242 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:46.243+0000 7f0c729e6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:55:46.287 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.288+0000 7fea81529040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T08:55:46.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.306+0000 7f8164963640 1 Processor -- start 2024-05-09T08:55:46.305 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.306+0000 7f8164963640 1 -- start start 2024-05-09T08:55:46.307 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8164963640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f816015b030 0x7f8160067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8164963640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8160073360 0x7f81600c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8164963640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600c91f0 con 0x7f816015b410 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8164963640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f816006ef20 con 0x7f8160073360 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8164963640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f81600c8f10 con 0x7f816015b030 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f815e59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f816015b030 0x7f8160067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8157fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f816015b410 0x7f8160154be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:56564/0 (socket says 172.21.15.183:56564) 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f8157fff640 1 -- 172.21.15.183:0/3135678542 learned_addr learned my addr 172.21.15.183:0/3135678542 (peer_addr_for_me v1:172.21.15.183:0/0) 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.307+0000 7f815ed9d640 1 --2- 172.21.15.183:0/3135678542 >> v2:172.21.15.49:3300/0 conn(0x7f8160073360 0x7f81600c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815e59c640 1 -- 172.21.15.183:0/3135678542 >> v2:172.21.15.49:3300/0 conn(0x7f8160073360 msgr2=0x7f81600c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815e59c640 1 --2- 172.21.15.183:0/3135678542 >> v2:172.21.15.49:3300/0 conn(0x7f8160073360 0x7f81600c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815e59c640 1 -- 172.21.15.183:0/3135678542 >> v1:172.21.15.183:6789/0 conn(0x7f816015b410 legacy=0x7f8160154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815e59c640 1 -- 172.21.15.183:0/3135678542 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816015b8f0 con 0x7f816015b030 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815dd9b640 1 -- 172.21.15.183:0/3135678542 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1700323611 0 0) 0x7f81600c91f0 con 0x7f816015b410 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815e59c640 1 --2- 172.21.15.183:0/3135678542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f816015b030 0x7f8160067a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81600ca560 tx=0x7f815002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d37714430ff27c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815dd9b640 1 -- 172.21.15.183:0/3135678542 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150037070 con 0x7f816015b030 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815dd9b640 1 -- 172.21.15.183:0/3135678542 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150032440 con 0x7f816015b030 2024-05-09T08:55:46.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.308+0000 7f815dd9b640 1 -- 172.21.15.183:0/3135678542 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150032810 con 0x7f816015b030 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 -- 172.21.15.183:0/3135678542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f816015b030 msgr2=0x7f8160067a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 --2- 172.21.15.183:0/3135678542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f816015b030 0x7f8160067a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81600ca560 tx=0x7f815002a230 comp rx=0 tx=0).stop 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 -- 172.21.15.183:0/3135678542 shutdown_connections 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 --2- 172.21.15.183:0/3135678542 >> v2:172.21.15.49:3300/0 conn(0x7f8160073360 0x7f81600c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 --2- 172.21.15.183:0/3135678542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f816015b030 0x7f8160067a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 -- 172.21.15.183:0/3135678542 >> 172.21.15.183:0/3135678542 conn(0x7f81600c9440 msgr2=0x7f81600c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.312 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 -- 172.21.15.183:0/3135678542 shutdown_connections 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.309+0000 7f8164963640 1 -- 172.21.15.183:0/3135678542 wait complete. 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.310+0000 7f8164963640 1 Processor -- start 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.310+0000 7f8164963640 1 -- start start 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f816015b030 0x7f8160159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f81601552f0 con 0x7f816015b030 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f816006ebb0 con 0x7f8160073360 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8164963640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8160155470 con 0x7f816015b410 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815ed9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.313 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57692/0 (socket says 172.21.15.183:57692) 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815ed9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34480/0 (socket says 172.21.15.183:34480) 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 -- 172.21.15.183:0/4162417907 learned_addr learned my addr 172.21.15.183:0/4162417907 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f8157fff640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f816015b030 0x7f8160159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 -- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 msgr2=0x7f81601e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.311+0000 7f815e59c640 1 -- 172.21.15.183:0/4162417907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f816015b030 msgr2=0x7f8160159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f815e59c640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f816015b030 0x7f8160159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f815e59c640 1 -- 172.21.15.183:0/4162417907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8160145620 con 0x7f8160073360 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f815ed9d640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f815e59c640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8150009890 tx=0x7f81500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150037040 con 0x7f8160073360 2024-05-09T08:55:46.314 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150042070 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150030bd0 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816014fba0 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81600cc1d0 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8150047460 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.312+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f815003da30 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.313+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816004f660 con 0x7f8160073360 2024-05-09T08:55:46.315 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.315+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8150032e20 con 0x7f8160073360 2024-05-09T08:55:46.359 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:46.360+0000 7f0c729e6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:55:46.385 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.386+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9177552d-bc69-409e-9654-e030c3a1afba", "id": 6} v 0) -- 0x7f81600c3840 con 0x7f8160073360 2024-05-09T08:55:46.388 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.389+0000 7f8156ffd640 1 -- 172.21.15.183:0/4162417907 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9177552d-bc69-409e-9654-e030c3a1afba", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8150030e10 con 0x7f8160073360 2024-05-09T08:55:46.389 INFO:teuthology.orchestra.run.smithi183.stdout:6 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 msgr2=0x7f81601514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8150009890 tx=0x7f81500046c0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 shutdown_connections 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f816015b410 0x7f81601e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f816015b030 0x7f8160159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.390 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 --2- 172.21.15.183:0/4162417907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8160073360 0x7f81601514d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 >> 172.21.15.183:0/4162417907 conn(0x7f81600c9440 msgr2=0x7f81600c7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 shutdown_connections 2024-05-09T08:55:46.391 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.390+0000 7f8164963640 1 -- 172.21.15.183:0/4162417907 wait complete. 2024-05-09T08:55:46.400 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new d3b1821c-1ed4-4286-b7de-030fda235fbf 7 2024-05-09T08:55:46.404 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.405+0000 7fea81529040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T08:55:46.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.538+0000 7efc5e0ee640 1 Processor -- start 2024-05-09T08:55:46.538 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.538+0000 7efc5e0ee640 1 -- start start 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efc580cd6e0 0x7efc580c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7efc580c9be0 con 0x7efc580cd6e0 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7efc580cdc50 con 0x7efc58151c00 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc5e0ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efc58157ac0 con 0x7efc580cd280 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc577fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc56ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc56ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34484/0 (socket says 172.21.15.183:34484) 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc577fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57702/0 (socket says 172.21.15.183:57702) 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc56ffd640 1 -- 172.21.15.183:0/3947800733 learned_addr learned my addr 172.21.15.183:0/3947800733 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:46.540 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.539+0000 7efc57fff640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.49:3300/0 conn(0x7efc580cd6e0 0x7efc580c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 -- 172.21.15.183:0/3947800733 >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 msgr2=0x7efc580c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 -- 172.21.15.183:0/3947800733 >> v2:172.21.15.49:3300/0 conn(0x7efc580cd6e0 msgr2=0x7efc580c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.49:3300/0 conn(0x7efc580cd6e0 0x7efc580c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 -- 172.21.15.183:0/3947800733 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc58157da0 con 0x7efc580cd280 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc577fe640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3947800733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc580caea0 tx=0x7efc4802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ec43c6eafbd68a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.540+0000 7efc567fc640 1 -- 172.21.15.183:0/3947800733 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48036020 con 0x7efc580cd280 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc567fc640 1 -- 172.21.15.183:0/3947800733 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc48032820 con 0x7efc580cd280 2024-05-09T08:55:46.541 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc567fc640 1 -- 172.21.15.183:0/3947800733 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48032bf0 con 0x7efc580cd280 2024-05-09T08:55:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3947800733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 msgr2=0x7efc580c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.544 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3947800733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc580caea0 tx=0x7efc4802c920 comp rx=0 tx=0).stop 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3947800733 shutdown_connections 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.49:3300/0 conn(0x7efc580cd6e0 0x7efc580c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3947800733 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc580c8180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3947800733 >> v2:172.21.15.183:3300/0 conn(0x7efc58151c00 0x7efc580c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3947800733 >> 172.21.15.183:0/3947800733 conn(0x7efc5806b200 msgr2=0x7efc5806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3947800733 shutdown_connections 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.541+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3947800733 wait complete. 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.542+0000 7efc5e0ee640 1 Processor -- start 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.542+0000 7efc5e0ee640 1 -- start start 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc58157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 0x7efc5815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 0x7efc58146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efc58158640 con 0x7efc58151c00 2024-05-09T08:55:46.545 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc58158360 con 0x7efc580cd6e0 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc5e0ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efc58151fe0 con 0x7efc580cd280 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc577fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc58157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 0x7efc5815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc577fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc58157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:34486/0 (socket says 172.21.15.183:34486) 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc577fe640 1 -- 172.21.15.183:0/3764516247 learned_addr learned my addr 172.21.15.183:0/3764516247 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc57fff640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 0x7efc58146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 -- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 msgr2=0x7efc58157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc58157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 -- 172.21.15.183:0/3764516247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 msgr2=0x7efc58146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 0x7efc58146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.543+0000 7efc56ffd640 1 -- 172.21.15.183:0/3764516247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5814d870 con 0x7efc580cd6e0 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc57fff640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 0x7efc58146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:55:46.546 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc56ffd640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 0x7efc5815a710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc48034000 tx=0x7efc480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48036040 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc48032e20 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5814e420 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc480453f0 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5814ddc0 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc4804c060 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.544+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7efc48044a00 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.545+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5804f660 con 0x7efc580cd6e0 2024-05-09T08:55:46.547 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.547+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efc48040070 con 0x7efc580cd6e0 2024-05-09T08:55:46.617 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.618+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d3b1821c-1ed4-4286-b7de-030fda235fbf", "id": 7} v 0) -- 0x7efc580c8d10 con 0x7efc580cd6e0 2024-05-09T08:55:46.623 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.624+0000 7efc3ffff640 1 -- 172.21.15.183:0/3764516247 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d3b1821c-1ed4-4286-b7de-030fda235fbf", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7efc4802f070 con 0x7efc580cd6e0 2024-05-09T08:55:46.624 INFO:teuthology.orchestra.run.smithi183.stdout:7 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 msgr2=0x7efc5815a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 0x7efc5815a710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efc48034000 tx=0x7efc480046c0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 shutdown_connections 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efc58151c00 0x7efc58146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc580cd6e0 0x7efc5815a710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.625+0000 7efc5e0ee640 1 --2- 172.21.15.183:0/3764516247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efc580cd280 0x7efc58157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.626+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 >> 172.21.15.183:0/3764516247 conn(0x7efc5806b200 msgr2=0x7efc580c7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.625 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.626+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 shutdown_connections 2024-05-09T08:55:46.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-09T08:55:46.626+0000 7efc5e0ee640 1 -- 172.21.15.183:0/3764516247 wait complete. 2024-05-09T08:55:46.636 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T08:55:46.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T08:55:46.638 INFO:tasks.ceph.osd.0:Started 2024-05-09T08:55:46.638 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T08:55:46.638 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T08:55:46.640 INFO:tasks.ceph.osd.1:Started 2024-05-09T08:55:46.641 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T08:55:46.641 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T08:55:46.643 INFO:tasks.ceph.osd.2:Started 2024-05-09T08:55:46.643 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T08:55:46.643 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T08:55:46.645 INFO:tasks.ceph.osd.3:Started 2024-05-09T08:55:46.645 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T08:55:46.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T08:55:46.648 INFO:tasks.ceph.osd.4:Started 2024-05-09T08:55:46.648 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T08:55:46.648 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T08:55:46.650 INFO:tasks.ceph.osd.5:Started 2024-05-09T08:55:46.651 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T08:55:46.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T08:55:46.653 INFO:tasks.ceph.osd.6:Started 2024-05-09T08:55:46.653 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T08:55:46.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T08:55:46.655 INFO:tasks.ceph.osd.7:Started 2024-05-09T08:55:46.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:46.773 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:46.774+0000 7f0c729e6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:55:46.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.820+0000 7f99466ee640 1 Processor -- start 2024-05-09T08:55:46.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.820+0000 7f99466ee640 1 -- start start 2024-05-09T08:55:46.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f99466ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9940152f50 0x7f9940072be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f99466ee640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99400ca5e0 0x7f99400c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f99466ee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f994015c350 con 0x7f99400caa00 2024-05-09T08:55:46.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f99466ee640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f994015c100 con 0x7f99400ca5e0 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f99466ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f994015cf60 con 0x7f9940152f50 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f993f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99400ca5e0 0x7f99400c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f9944c8b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f99400caa00 0x7f9940158d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49892/0 (socket says 172.21.15.49:49892) 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.821+0000 7f9944c8b640 1 -- 172.21.15.49:0/50662574 learned_addr learned my addr 172.21.15.49:0/50662574 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.822+0000 7f993ffff640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9940152f50 0x7f9940072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.822+0000 7f993f7fe640 1 -- 172.21.15.49:0/50662574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9940152f50 msgr2=0x7f9940072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.822+0000 7f993f7fe640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9940152f50 0x7f9940072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.822+0000 7f993f7fe640 1 -- 172.21.15.49:0/50662574 >> v1:172.21.15.49:6789/0 conn(0x7f99400caa00 legacy=0x7f9940158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.822+0000 7f993f7fe640 1 -- 172.21.15.49:0/50662574 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994015c630 con 0x7f99400ca5e0 2024-05-09T08:55:46.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f993f7fe640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99400ca5e0 0x7f99400c5990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99340097a0 tx=0x7f993402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce1a08b2e62ebd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f993effd640 1 -- 172.21.15.49:0/50662574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934037070 con 0x7f99400ca5e0 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f993effd640 1 -- 172.21.15.49:0/50662574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993402ae50 con 0x7f99400ca5e0 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f993effd640 1 -- 172.21.15.49:0/50662574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934032660 con 0x7f99400ca5e0 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 -- 172.21.15.49:0/50662574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99400ca5e0 msgr2=0x7f99400c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99400ca5e0 0x7f99400c5990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99340097a0 tx=0x7f993402a100 comp rx=0 tx=0).stop 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 -- 172.21.15.49:0/50662574 shutdown_connections 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99400ca5e0 0x7f99400c5990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 --2- 172.21.15.49:0/50662574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9940152f50 0x7f9940072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 -- 172.21.15.49:0/50662574 >> 172.21.15.49:0/50662574 conn(0x7f9940067d60 msgr2=0x7f9940074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.823+0000 7f99466ee640 1 -- 172.21.15.49:0/50662574 shutdown_connections 2024-05-09T08:55:46.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.824+0000 7f99466ee640 1 -- 172.21.15.49:0/50662574 wait complete. 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.825+0000 7f99466ee640 1 Processor -- start 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.825+0000 7f99466ee640 1 -- start start 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99400caa00 0x7f99400cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 0x7f9940147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9940073120 con 0x7f9940152f50 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f994015cbf0 con 0x7f99400caa00 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f99466ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9940159440 con 0x7f99400ca5e0 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f993ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f9944c8b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 0x7f9940147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f993ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:37496/0 (socket says 172.21.15.49:37496) 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f993ffff640 1 -- 172.21.15.49:0/2231856291 learned_addr learned my addr 172.21.15.49:0/2231856291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.826+0000 7f993f7fe640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99400caa00 0x7f99400cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 -- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99400caa00 msgr2=0x7f99400cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99400caa00 0x7f99400cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 -- 172.21.15.49:0/2231856291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 msgr2=0x7f9940147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 0x7f9940147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 -- 172.21.15.49:0/2231856291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99400cce70 con 0x7f99400ca5e0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f9944c8b640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 0x7f9940147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993ffff640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f993000e870 tx=0x7f993000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930017070 con 0x7f99400ca5e0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993000c940 con 0x7f99400ca5e0 2024-05-09T08:55:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000cd30 con 0x7f99400ca5e0 2024-05-09T08:55:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99400cc320 con 0x7f99400ca5e0 2024-05-09T08:55:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f994015b4f0 con 0x7f99400ca5e0 2024-05-09T08:55:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f993001a070 con 0x7f99400ca5e0 2024-05-09T08:55:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.827+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9930006020 con 0x7f99400ca5e0 2024-05-09T08:55:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.828+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9940068c70 con 0x7f99400ca5e0 2024-05-09T08:55:46.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.831+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f99400cce70 con 0x7f99400ca5e0 2024-05-09T08:55:46.897 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:46.898+0000 7fea81529040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T08:55:46.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.918+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99400733d0 con 0x7f99400ca5e0 2024-05-09T08:55:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.919+0000 7f993cff9640 1 -- 172.21.15.49:0/2231856291 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f99300064d0 con 0x7f99400ca5e0 2024-05-09T08:55:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:46.919 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:55:46.620125+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 msgr2=0x7f99401f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f993000e870 tx=0x7f993000ed30 comp rx=0 tx=0).stop 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 shutdown_connections 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9940152f50 0x7f9940147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99400caa00 0x7f99400cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 --2- 172.21.15.49:0/2231856291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99400ca5e0 0x7f99401f23d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 >> 172.21.15.49:0/2231856291 conn(0x7f9940067d60 msgr2=0x7f99400c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 shutdown_connections 2024-05-09T08:55:46.920 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:46.920+0000 7f99466ee640 1 -- 172.21.15.49:0/2231856291 wait complete. 2024-05-09T08:55:46.930 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T08:55:46.930 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T08:55:47.029 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.030+0000 7f0c729e6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:55:47.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.081+0000 7f0c729e6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:55:47.170 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.170+0000 7fea81529040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T08:55:47.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.195+0000 7f0c729e6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:55:47.224 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.225+0000 7fea81529040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T08:55:47.251 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:55:47.252+0000 7f79b2fcb640 -1 Falling back to public interface 2024-05-09T08:55:47.258 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:55:47.259+0000 7f5decda3640 -1 Falling back to public interface 2024-05-09T08:55:47.262 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:55:47.262+0000 7f637d429640 -1 Falling back to public interface 2024-05-09T08:55:47.262 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:55:47.263+0000 7f18cf81c640 -1 Falling back to public interface 2024-05-09T08:55:47.265 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:55:47.265+0000 7f22c6d53640 -1 Falling back to public interface 2024-05-09T08:55:47.275 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:55:47.276+0000 7f030c229640 -1 Falling back to public interface 2024-05-09T08:55:47.277 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:55:47.278+0000 7f229b3b1640 -1 Falling back to public interface 2024-05-09T08:55:47.281 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:55:47.282+0000 7f0bc84e8640 -1 Falling back to public interface 2024-05-09T08:55:47.349 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.349+0000 7fea81529040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T08:55:47.474 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.475+0000 7f0c729e6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:55:47.521 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.522+0000 7f0c729e6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:55:47.653 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.654+0000 7fea81529040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T08:55:47.675 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.676+0000 7f0c729e6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:55:47.703 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.704+0000 7fea81529040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T08:55:47.735 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:47.809 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.810+0000 7f0c729e6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:55:47.830 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.831+0000 7fc7878da640 1 Processor -- start 2024-05-09T08:55:47.831 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 -- start start 2024-05-09T08:55:47.834 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc780158db0 0x7fc78006d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.834 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc78006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.834 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7800c8ef0 con 0x7fc780153050 2024-05-09T08:55:47.834 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc7800c8cc0 con 0x7fc780158db0 2024-05-09T08:55:47.834 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.832+0000 7fc7878da640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc7800c9790 con 0x7fc78006f980 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc785676640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fc780153050 0x7fc78015ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:40248/0 (socket says 172.21.15.49:40248) 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc785676640 1 -- 172.21.15.49:0/2375279873 learned_addr learned my addr 172.21.15.49:0/2375279873 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408383853 0 0) 0x7fc7800c8ef0 con 0x7fc780153050 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc76c003520 con 0x7fc780153050 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc784e75640 1 --2- 172.21.15.49:0/2375279873 >> v2:172.21.15.183:3300/0 conn(0x7fc780158db0 0x7fc78006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc785e77640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc78006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3906256217 0 0) 0x7fc76c003520 con 0x7fc780153050 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 msgr2=0x7fc78006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc78006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 >> v2:172.21.15.183:3300/0 conn(0x7fc780158db0 msgr2=0x7fc78006d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 --2- 172.21.15.49:0/2375279873 >> v2:172.21.15.183:3300/0 conn(0x7fc780158db0 0x7fc78006d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800c91d0 con 0x7fc780153050 2024-05-09T08:55:47.835 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fc77c001670 con 0x7fc780153050 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 >> v1:172.21.15.183:6789/0 conn(0x7fc780153050 legacy=0x7fc78015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc76c0039b0 0x7fc76c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc76c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c004d40 0x7fc76c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800c9790 con 0x7fc76c004d40 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c00bad0 con 0x7fc76c0039b0 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c00bcd0 con 0x7fc78006f980 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc785676640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc76c0039b0 0x7fc76c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc785e77640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c004d40 0x7fc76c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.833+0000 7fc784e75640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc76c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 -- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 msgr2=0x7fc76c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78006f980 0x7fc76c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 -- 172.21.15.49:0/2375279873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c004d40 msgr2=0x7fc76c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:47.836 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c004d40 0x7fc76c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.837 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 -- 172.21.15.49:0/2375279873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c003520 con 0x7fc76c0039b0 2024-05-09T08:55:47.837 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.834+0000 7fc785676640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc76c0039b0 0x7fc76c003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c210 tx=0x7fc76c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dbf859d5abcfaaa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:47.837 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007d50 con 0x7fc76c0039b0 2024-05-09T08:55:47.837 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c005af0 con 0x7fc76c0039b0 2024-05-09T08:55:47.837 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc777fff640 1 -- 172.21.15.49:0/2375279873 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007070 con 0x7fc76c0039b0 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 -- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc76c0039b0 msgr2=0x7fc76c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 --2- 172.21.15.49:0/2375279873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc76c0039b0 0x7fc76c003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c210 tx=0x7fc76c005830 comp rx=0 tx=0).stop 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc785e77640 1 -- 172.21.15.49:0/2375279873 reap_dead start 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 -- 172.21.15.49:0/2375279873 shutdown_connections 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 -- 172.21.15.49:0/2375279873 >> 172.21.15.49:0/2375279873 conn(0x7fc78006e6b0 msgr2=0x7fc78006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 -- 172.21.15.49:0/2375279873 shutdown_connections 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.835+0000 7fc7878da640 1 -- 172.21.15.49:0/2375279873 wait complete. 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.836+0000 7fc7878da640 1 Processor -- start 2024-05-09T08:55:47.840 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 -- start start 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7800c5360 0x7fc7800c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78015a330 0x7fc78015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc78015c4d0 con 0x7fc76c0039b0 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc78015c840 con 0x7fc7800c5360 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc7878da640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc780153430 con 0x7fc78015a330 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc785676640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc785676640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49824/0 (socket says 172.21.15.49:49824) 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc785676640 1 -- 172.21.15.49:0/971318355 learned_addr learned my addr 172.21.15.49:0/971318355 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc785e77640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78015a330 0x7fc78015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.837+0000 7fc784e75640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7800c5360 0x7fc7800c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 -- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78015a330 msgr2=0x7fc78015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78015a330 0x7fc78015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.841 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 -- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7800c5360 msgr2=0x7fc7800c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7800c5360 0x7fc7800c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 -- 172.21.15.49:0/971318355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78015acf0 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc785676640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c210 tx=0x7fc77c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c042070 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c007480 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007d50 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7800cc1c0 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7800c5e20 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.838+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc77c045070 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.839+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc77c03eb20 con 0x7fc76c0039b0 2024-05-09T08:55:47.842 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.839+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc780068e90 con 0x7fc76c0039b0 2024-05-09T08:55:47.843 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.841+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc78015acf0 con 0x7fc76c0039b0 2024-05-09T08:55:47.860 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.861+0000 7f0c729e6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:55:47.872 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:47.872+0000 7fea81529040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T08:55:47.903 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.904+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7800c7fc0 con 0x7fc76c0039b0 2024-05-09T08:55:47.904 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.904+0000 7fc775ffb640 1 -- 172.21.15.49:0/971318355 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc77c005a40 con 0x7fc76c0039b0 2024-05-09T08:55:47.904 INFO:teuthology.misc.health.smithi049.stdout: 2024-05-09T08:55:47.904 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:55:46.620125+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 msgr2=0x7fc78015bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c210 tx=0x7fc77c001a40 comp rx=0 tx=0).stop 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 shutdown_connections 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc78015a330 0x7fc78015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7800c5360 0x7fc7800c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 --2- 172.21.15.49:0/971318355 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc76c0039b0 0x7fc78015bb50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 >> 172.21.15.49:0/971318355 conn(0x7fc78006e6b0 msgr2=0x7fc78014ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 shutdown_connections 2024-05-09T08:55:47.905 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:47.905+0000 7fc7878da640 1 -- 172.21.15.49:0/971318355 wait complete. 2024-05-09T08:55:47.914 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T08:55:47.928 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.929+0000 7f0c729e6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:55:47.972 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:47.973+0000 7f0c729e6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:55:48.017 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.018+0000 7fea81529040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T08:55:48.069 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.070+0000 7fea81529040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T08:55:48.143 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.144+0000 7fea81529040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T08:55:48.190 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.191+0000 7fea81529040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T08:55:48.444 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:48.445+0000 7f0c729e6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:55:48.558 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:48.559+0000 7f0c729e6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:55:48.695 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.695+0000 7fea81529040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T08:55:48.815 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-05-09T08:55:48.816+0000 7fea81529040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T08:55:49.574 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:49.575+0000 7f0c13dd9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T08:55:49.574 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-09T08:55:49.575+0000 7f0c13dd9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T08:55:50.734 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:55:50.735+0000 7f22c6d53640 -1 osd.7 0 log_to_monitors true 2024-05-09T08:55:50.736 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:55:50.737+0000 7f5decda3640 -1 osd.0 0 log_to_monitors true 2024-05-09T08:55:50.744 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:55:50.746+0000 7f637d429640 -1 osd.4 0 log_to_monitors true 2024-05-09T08:55:50.758 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:55:50.759+0000 7f18cf81c640 -1 osd.2 0 log_to_monitors true 2024-05-09T08:55:50.771 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:55:50.772+0000 7f79b2fcb640 -1 osd.5 0 log_to_monitors true 2024-05-09T08:55:50.775 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:55:50.776+0000 7f030c229640 -1 osd.1 0 log_to_monitors true 2024-05-09T08:55:50.793 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:55:50.794+0000 7f229b3b1640 -1 osd.3 0 log_to_monitors true 2024-05-09T08:55:50.798 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:55:50.799+0000 7f0bc84e8640 -1 osd.6 0 log_to_monitors true 2024-05-09T08:55:54.013 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:55:54.014+0000 7f2296b24640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T08:55:54.026 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:55:54.027+0000 7f228f156640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.075 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:55:54.076+0000 7f18ca97d640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T08:55:54.086 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:55:54.087+0000 7f18c2faf640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.159 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:55:54.160+0000 7f5de8516640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T08:55:54.173 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:55:54.174+0000 7f030799c640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T08:55:54.177 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:55:54.178+0000 7f5de0b48640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.185 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:55:54.186+0000 7f02fffce640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.554 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:55:54.555+0000 7f0bc3c5b640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T08:55:54.565 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:55:54.566+0000 7f6378b9c640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T08:55:54.571 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:55:54.572+0000 7f0bbc28d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.577 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:55:54.578+0000 7f63711ce640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.605 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:55:54.606+0000 7f79ae73e640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T08:55:54.615 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:55:54.616+0000 7f79a6d70640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.637 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:55:54.638+0000 7f22c24c6640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T08:55:54.657 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:55:54.658+0000 7f22baaf8640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:55:54.721 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:54.818 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.819+0000 7f2514d1b640 1 Processor -- start 2024-05-09T08:55:54.818 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.819+0000 7f2514d1b640 1 -- start start 2024-05-09T08:55:54.821 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2514d1b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f251015b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2514d1b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f25100cb480 0x7f25100c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2514d1b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f251015e3d0 con 0x7f25100c5b10 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2514d1b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2510155310 con 0x7f25100cb480 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2514d1b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f251015e670 con 0x7f2510154f30 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250ed9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f25100c5b10 0x7f25100c6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58398/0 (socket says 172.21.15.49:58398) 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250ed9d640 1 -- 172.21.15.49:0/1076006815 learned_addr learned my addr 172.21.15.49:0/1076006815 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3495938491 0 0) 0x7f251015e3d0 con 0x7f25100c5b10 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250e59c640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f251015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24fc003520 con 0x7f25100c5b10 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f2507fff640 1 --2- 172.21.15.49:0/1076006815 >> v2:172.21.15.183:3300/0 conn(0x7f25100cb480 0x7f25100c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2332448409 0 0) 0x7f24fc003520 con 0x7f25100c5b10 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 msgr2=0x7f251015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:54.822 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f251015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 >> v2:172.21.15.183:3300/0 conn(0x7f25100cb480 msgr2=0x7f25100c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 --2- 172.21.15.49:0/1076006815 >> v2:172.21.15.183:3300/0 conn(0x7f25100cb480 0x7f25100c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251015e950 con 0x7f25100c5b10 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f25080019f0 con 0x7f25100c5b10 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 >> v1:172.21.15.49:6789/0 conn(0x7f25100c5b10 legacy=0x7f25100c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f24fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc003f30 0x7f24fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24fc004d40 0x7f24fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251015e670 con 0x7f24fc004d40 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc00b9c0 con 0x7f24fc003f30 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.820+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc00bb70 con 0x7f2510154f30 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24fc004d40 0x7f24fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250e59c640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f24fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f2507fff640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc003f30 0x7f24fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.823 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 -- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 msgr2=0x7f24fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2510154f30 0x7f24fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 -- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc003f30 msgr2=0x7f24fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc003f30 0x7f24fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 -- 172.21.15.49:0/1076006815 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc003520 con 0x7f24fc004d40 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.821+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24fc004d40 0x7f24fc0051c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f250800a870 tx=0x7f24fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d703328ecd1b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800a4c0 con 0x7f24fc004d40 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250802f070 con 0x7f24fc004d40 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f250dd9b640 1 -- 172.21.15.49:0/1076006815 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250802bcc0 con 0x7f24fc004d40 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 -- 172.21.15.49:0/1076006815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24fc004d40 msgr2=0x7f24fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1076006815 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24fc004d40 0x7f24fc0051c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f250800a870 tx=0x7f24fc005830 comp rx=0 tx=0).stop 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f250ed9d640 1 -- 172.21.15.49:0/1076006815 reap_dead start 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 -- 172.21.15.49:0/1076006815 shutdown_connections 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 -- 172.21.15.49:0/1076006815 >> 172.21.15.49:0/1076006815 conn(0x7f251006cb10 msgr2=0x7f251006cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 -- 172.21.15.49:0/1076006815 shutdown_connections 2024-05-09T08:55:54.824 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.822+0000 7f2514d1b640 1 -- 172.21.15.49:0/1076006815 wait complete. 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 Processor -- start 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 -- start start 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc004d40 0x7f25100b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f251015b940 0x7f25101463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f251015f1f0 con 0x7f251015c110 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f251015ef10 con 0x7f24fc004d40 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f2514d1b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f25100c6210 con 0x7f251015b940 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f250ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f250ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58500/0 (socket says 172.21.15.49:58500) 2024-05-09T08:55:54.827 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.824+0000 7f250ed9d640 1 -- 172.21.15.49:0/1285257331 learned_addr learned my addr 172.21.15.49:0/1285257331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2507fff640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f251015b940 0x7f25101463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250e59c640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc004d40 0x7f25100b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 -- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f251015b940 msgr2=0x7f25101463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f251015b940 0x7f25101463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 -- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc004d40 msgr2=0x7f25100b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc004d40 0x7f25100b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 -- 172.21.15.49:0/1285257331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251015cad0 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f250ed9d640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f250800a990 tx=0x7f2508001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800a4c0 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25080019e0 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25100b5880 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508002030 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.825+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25100b3440 con 0x7f251015c110 2024-05-09T08:55:54.828 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.826+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2510068e90 con 0x7f251015c110 2024-05-09T08:55:54.829 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.827+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f2508005020 con 0x7f251015c110 2024-05-09T08:55:54.829 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.827+0000 7f2506ffd640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 0x7f24f006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:54.829 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.827+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f25080a9c40 con 0x7f251015c110 2024-05-09T08:55:54.829 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.827+0000 7f250e59c640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 0x7f24f006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:54.829 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.828+0000 7f250e59c640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 0x7f24f006e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2500009bc0 tx=0x7f2500009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:54.831 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:54.832+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f25080b0080 con 0x7f251015c110 2024-05-09T08:55:55.018 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.018+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f251014b8e0 con 0x7f251015c110 2024-05-09T08:55:55.020 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.021+0000 7f2506ffd640 1 -- 172.21.15.49:0/1285257331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10897 (secure 0 0 0) 0x7f251015cad0 con 0x7f251015c110 2024-05-09T08:55:55.020 INFO:teuthology.misc.health.smithi049.stdout: 2024-05-09T08:55:55.021 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":13,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:55:55.016806+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 msgr2=0x7f24f006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 0x7f24f006e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2500009bc0 tx=0x7f2500009190 comp rx=0 tx=0).stop 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 msgr2=0x7f251015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f250800a990 tx=0x7f2508001360 comp rx=0 tx=0).stop 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 shutdown_connections 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24f006bda0 0x7f24f006e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.025 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f251015c110 0x7f251015c590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.026 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f251015b940 0x7f25101463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.026 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 --2- 172.21.15.49:0/1285257331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24fc004d40 0x7f25100b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.026 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 >> 172.21.15.49:0/1285257331 conn(0x7f251006cb10 msgr2=0x7f25100cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.026 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 shutdown_connections 2024-05-09T08:55:55.026 INFO:teuthology.misc.health.smithi049.stderr:2024-05-09T08:55:55.025+0000 7f2514d1b640 1 -- 172.21.15.49:0/1285257331 wait complete. 2024-05-09T08:55:55.035 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T08:55:55.035 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T08:55:55.035 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T08:55:55.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T08:55:55.037 INFO:tasks.ceph.mds.a:Started 2024-05-09T08:55:55.037 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T08:55:55.038 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T08:55:55.039 INFO:tasks.ceph.mds.c:Started 2024-05-09T08:55:55.040 INFO:tasks.ceph.mds.e:Restarting daemon 2024-05-09T08:55:55.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-05-09T08:55:55.042 INFO:tasks.ceph.mds.e:Started 2024-05-09T08:55:55.042 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T08:55:55.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T08:55:55.044 INFO:tasks.ceph.mds.b:Started 2024-05-09T08:55:55.044 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T08:55:55.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T08:55:55.047 INFO:tasks.ceph.mds.d:Started 2024-05-09T08:55:55.047 INFO:tasks.ceph.mds.f:Restarting daemon 2024-05-09T08:55:55.047 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-05-09T08:55:55.049 INFO:tasks.ceph.mds.f:Started 2024-05-09T08:55:55.049 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T08:55:55.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:55.118 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2024-05-09T08:55:55.119 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-05-09T08:55:55.119 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2024-05-09T08:55:55.119 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2024-05-09T08:55:55.120 INFO:tasks.ceph.mds.f.smithi183.stdout:starting mds.f at 2024-05-09T08:55:55.121 INFO:tasks.ceph.mds.e.smithi049.stdout:starting mds.e at 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.162+0000 7f750e5ed640 1 Processor -- start 2024-05-09T08:55:55.162 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 -- start start 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7508075200 0x7f750815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75080cc190 0x7f75080cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75080ccb60 con 0x7f75081517f0 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7508151bd0 con 0x7f75080cc190 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.163+0000 7f750e5ed640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f750815e260 con 0x7f7508075200 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f750cb8a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75080cc190 0x7f75080cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f7507fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f75081517f0 0x7f75080ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58430/0 (socket says 172.21.15.49:58430) 2024-05-09T08:55:55.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f7507fff640 1 -- 172.21.15.49:0/3127155589 learned_addr learned my addr 172.21.15.49:0/3127155589 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f750cb8a640 1 -- 172.21.15.49:0/3127155589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7508075200 msgr2=0x7f750815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f75077fe640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7508075200 0x7f750815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f750cb8a640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7508075200 0x7f750815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f750cb8a640 1 -- 172.21.15.49:0/3127155589 >> v1:172.21.15.49:6789/0 conn(0x7f75081517f0 legacy=0x7f75080ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.164+0000 7f750cb8a640 1 -- 172.21.15.49:0/3127155589 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750815e520 con 0x7f75080cc190 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750cb8a640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f75080cc620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7500009dc0 tx=0x7f750002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f56d3dd9dbcb51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f7506ffd640 1 -- 172.21.15.49:0/3127155589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500007740 con 0x7f75080cc190 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f7506ffd640 1 -- 172.21.15.49:0/3127155589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75000078e0 con 0x7f75080cc190 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f7506ffd640 1 -- 172.21.15.49:0/3127155589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500007cb0 con 0x7f75080cc190 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 -- 172.21.15.49:0/3127155589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 msgr2=0x7f75080cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f75080cc620 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7500009dc0 tx=0x7f750002a710 comp rx=0 tx=0).stop 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 -- 172.21.15.49:0/3127155589 shutdown_connections 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f75080cc620 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 --2- 172.21.15.49:0/3127155589 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7508075200 0x7f750815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 -- 172.21.15.49:0/3127155589 >> 172.21.15.49:0/3127155589 conn(0x7f750806bd00 msgr2=0x7f7508074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 -- 172.21.15.49:0/3127155589 shutdown_connections 2024-05-09T08:55:55.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.165+0000 7f750e5ed640 1 -- 172.21.15.49:0/3127155589 wait complete. 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.166+0000 7f750e5ed640 1 Processor -- start 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 -- start start 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 0x7f7508144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75081517f0 0x7f750815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f75077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f75077fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58558/0 (socket says 172.21.15.49:58558) 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f75077fe640 1 -- 172.21.15.49:0/2400986190 learned_addr learned my addr 172.21.15.49:0/2400986190 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f750815edc0 con 0x7f75080cc190 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f750815eae0 con 0x7f7508075200 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.167+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f750815f0a0 con 0x7f75081517f0 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f750cb8a640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75081517f0 0x7f750815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f7507fff640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 0x7f7508144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 -- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75081517f0 msgr2=0x7f750815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75081517f0 0x7f750815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 -- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 msgr2=0x7f7508144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 0x7f7508144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7508144e20 con 0x7f75080cc190 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f7507fff640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 0x7f7508144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f75077fe640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00c7f0 tx=0x7f74fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc015070 con 0x7f75080cc190 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc004240 con 0x7f75080cc190 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc007620 con 0x7f75080cc190 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7508144a80 con 0x7f75080cc190 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.168+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508158eb0 con 0x7f75080cc190 2024-05-09T08:55:55.170 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.169+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7508068560 con 0x7f75080cc190 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.169+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f74fc0079d0 con 0x7f75080cc190 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.170+0000 7f7504ff9640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 0x7f74f406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.170+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f74fc08abe0 con 0x7f75080cc190 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.171+0000 7f7507fff640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 0x7f74f406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.171+0000 7f7507fff640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 0x7f74f406e260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74f8000f60 tx=0x7f74f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.174+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74fc059bd0 con 0x7f75080cc190 2024-05-09T08:55:55.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.358+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7508148180 con 0x7f75080cc190 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.359+0000 7f7504ff9640 1 -- 172.21.15.49:0/2400986190 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10897 (secure 0 0 0) 0x7f74fc0594d0 con 0x7f75080cc190 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:55.358 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":13,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:55:55.016806+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 msgr2=0x7f74f406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 0x7f74f406e260 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74f8000f60 tx=0x7f74f8009040 comp rx=0 tx=0).stop 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 msgr2=0x7f7508159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00c7f0 tx=0x7f74fc00ccb0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 shutdown_connections 2024-05-09T08:55:55.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.360+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74f406bda0 0x7f74f406e260 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75081517f0 0x7f750815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75080cc190 0x7f7508159f50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 --2- 172.21.15.49:0/2400986190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7508075200 0x7f7508144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 >> 172.21.15.49:0/2400986190 conn(0x7f750806bd00 msgr2=0x7f7508074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 shutdown_connections 2024-05-09T08:55:55.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.361+0000 7f750e5ed640 1 -- 172.21.15.49:0/2400986190 wait complete. 2024-05-09T08:55:55.371 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-05-09T08:55:55.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.476+0000 7f580ca38640 1 Processor -- start 2024-05-09T08:55:55.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.476+0000 7f580ca38640 1 -- start start 2024-05-09T08:55:55.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f580ca38640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58080c57f0 0x7f58080b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f580ca38640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f58080d08e0 0x7f5808165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f580ca38640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58080cb520 con 0x7f58080d0500 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f580ca38640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f58080cb800 con 0x7f58080c57f0 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f580ca38640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f580815e400 con 0x7f58080d08e0 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58080c57f0 0x7f58080b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f58080c57f0 0x7f58080b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58570/0 (socket says 172.21.15.49:58570) 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 -- 172.21.15.49:0/3109694093 learned_addr learned my addr 172.21.15.49:0/3109694093 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 -- 172.21.15.49:0/3109694093 >> v2:172.21.15.183:3300/0 conn(0x7f58080d08e0 msgr2=0x7f5808165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 --2- 172.21.15.49:0/3109694093 >> v2:172.21.15.183:3300/0 conn(0x7f58080d08e0 0x7f5808165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 -- 172.21.15.49:0/3109694093 >> v1:172.21.15.183:6789/0 conn(0x7f58080d0500 legacy=0x7f580815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:55.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.477+0000 7f5805d9b640 1 -- 172.21.15.49:0/3109694093 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580815e6e0 con 0x7f58080c57f0 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.478+0000 7f5805d9b640 1 --2- 172.21.15.49:0/3109694093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080c57f0 0x7f58080b3370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f580806c980 tx=0x7f57fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587e5ba43d6d0c6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.478+0000 7f580559a640 1 -- 172.21.15.49:0/3109694093 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc037070 con 0x7f58080c57f0 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.478+0000 7f580559a640 1 -- 172.21.15.49:0/3109694093 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc032440 con 0x7f58080c57f0 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.478+0000 7f580559a640 1 -- 172.21.15.49:0/3109694093 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc032810 con 0x7f58080c57f0 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 -- 172.21.15.49:0/3109694093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080c57f0 msgr2=0x7f58080b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 --2- 172.21.15.49:0/3109694093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080c57f0 0x7f58080b3370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f580806c980 tx=0x7f57fc02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 -- 172.21.15.49:0/3109694093 shutdown_connections 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 --2- 172.21.15.49:0/3109694093 >> v2:172.21.15.183:3300/0 conn(0x7f58080d08e0 0x7f5808165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 --2- 172.21.15.49:0/3109694093 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080c57f0 0x7f58080b3370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 -- 172.21.15.49:0/3109694093 >> 172.21.15.49:0/3109694093 conn(0x7f5808067940 msgr2=0x7f5808072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 -- 172.21.15.49:0/3109694093 shutdown_connections 2024-05-09T08:55:55.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.479+0000 7f580ca38640 1 -- 172.21.15.49:0/3109694093 wait complete. 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.480+0000 7f580ca38640 1 Processor -- start 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.480+0000 7f580ca38640 1 -- start start 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58080c57f0 0x7f580815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58080d0500 0x7f580814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f58080cca40 con 0x7f58080d08e0 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f580815bbe0 con 0x7f58080d0500 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580ca38640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f58080c5d40 con 0x7f58080c57f0 2024-05-09T08:55:55.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58582/0 (socket says 172.21.15.49:58582) 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 -- 172.21.15.49:0/3822562493 learned_addr learned my addr 172.21.15.49:0/3822562493 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5805d9b640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58080d0500 0x7f580814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f580659c640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58080c57f0 0x7f580815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 -- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58080c57f0 msgr2=0x7f580815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58080c57f0 0x7f580815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 -- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58080d0500 msgr2=0x7f580814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58080d0500 0x7f580814e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 -- 172.21.15.49:0/3822562493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5808159ec0 con 0x7f58080d08e0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.481+0000 7f5806d9d640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f580000b970 tx=0x7f580000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.482+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800019070 con 0x7f58080d08e0 2024-05-09T08:55:55.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.482+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580000cb60 con 0x7f58080d08e0 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.482+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5800014440 con 0x7f58080d08e0 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.482+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580815aea0 con 0x7f58080d08e0 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.482+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580815a160 con 0x7f58080d08e0 2024-05-09T08:55:55.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.483+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5808068560 con 0x7f58080d08e0 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.483+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f580001c070 con 0x7f58080d08e0 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.484+0000 7f57f6ffd640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 0x7f57e806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.484+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f580008a5b0 con 0x7f58080d08e0 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.484+0000 7f580659c640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 0x7f57e806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:55.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.484+0000 7f580659c640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 0x7f57e806e260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57f80017f0 tx=0x7f57f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:55.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.488+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5800058e40 con 0x7f58080d08e0 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.712+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58080cb260 con 0x7f58080d08e0 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.712+0000 7f57f6ffd640 1 -- 172.21.15.49:0/3822562493 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f580005d140 con 0x7f58080d08e0 2024-05-09T08:55:55.711 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T08:55:55.712 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 2 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 msgr2=0x7f57e806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 0x7f57e806e260 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f57f80017f0 tx=0x7f57f8009040 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 msgr2=0x7f580814dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f580000b970 tx=0x7f580000be30 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 shutdown_connections 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f57e806bda0 0x7f57e806e260 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f58080d08e0 0x7f580814dab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58080d0500 0x7f580814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 --2- 172.21.15.49:0/3822562493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58080c57f0 0x7f580815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 >> 172.21.15.49:0/3822562493 conn(0x7f5808067940 msgr2=0x7f58080ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:55.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 shutdown_connections 2024-05-09T08:55:55.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:55.714+0000 7f580ca38640 1 -- 172.21.15.49:0/3822562493 wait complete. 2024-05-09T08:55:57.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:55:57.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.845+0000 7fb37dcc1640 1 Processor -- start 2024-05-09T08:55:57.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.845+0000 7fb37dcc1640 1 -- start start 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb37dcc1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 0x7fb37815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb37dcc1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 0x7fb3780cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb37dcc1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3780ccb60 con 0x7fb378075200 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb37dcc1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb378151bd0 con 0x7fb3781517f0 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb37dcc1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb37815e260 con 0x7fb3780cc190 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb376ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb378075200 0x7fb3780ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58442/0 (socket says 172.21.15.49:58442) 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb376ffd640 1 -- 172.21.15.49:0/3755986687 learned_addr learned my addr 172.21.15.49:0/3755986687 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb3777fe640 1 --2- 172.21.15.49:0/3755986687 >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 0x7fb37815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb377fff640 1 --2- 172.21.15.49:0/3755986687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 0x7fb3780cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb3767fc640 1 -- 172.21.15.49:0/3755986687 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007756149 0 0) 0x7fb3780ccb60 con 0x7fb378075200 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb3767fc640 1 -- 172.21.15.49:0/3755986687 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb360003520 con 0x7fb378075200 2024-05-09T08:55:57.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb377fff640 1 -- 172.21.15.49:0/3755986687 >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 msgr2=0x7fb37815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:57.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb377fff640 1 --2- 172.21.15.49:0/3755986687 >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 0x7fb37815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb377fff640 1 -- 172.21.15.49:0/3755986687 >> v1:172.21.15.49:6789/0 conn(0x7fb378075200 legacy=0x7fb3780ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:57.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb377fff640 1 -- 172.21.15.49:0/3755986687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37815e520 con 0x7fb3780cc190 2024-05-09T08:55:57.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.846+0000 7fb3777fe640 1 --2- 172.21.15.49:0/3755986687 >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 0x7fb37815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.847+0000 7fb377fff640 1 --2- 172.21.15.49:0/3755986687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 0x7fb3780cc620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb370009d90 tx=0x7fb37002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8fb68375f26943 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.847+0000 7fb3767fc640 1 -- 172.21.15.49:0/3755986687 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007740 con 0x7fb3780cc190 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.847+0000 7fb3767fc640 1 -- 172.21.15.49:0/3755986687 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3700078e0 con 0x7fb3780cc190 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.847+0000 7fb3767fc640 1 -- 172.21.15.49:0/3755986687 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370007cb0 con 0x7fb3780cc190 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3755986687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 msgr2=0x7fb3780cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3755986687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 0x7fb3780cc620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb370009d90 tx=0x7fb37002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3755986687 shutdown_connections 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3755986687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb3780cc190 0x7fb3780cc620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3755986687 >> v2:172.21.15.183:3300/0 conn(0x7fb3781517f0 0x7fb37815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3755986687 >> 172.21.15.49:0/3755986687 conn(0x7fb37806bd00 msgr2=0x7fb378074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:57.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3755986687 shutdown_connections 2024-05-09T08:55:57.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.848+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3755986687 wait complete. 2024-05-09T08:55:57.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.850+0000 7fb37dcc1640 1 Processor -- start 2024-05-09T08:55:57.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.850+0000 7fb37dcc1640 1 -- start start 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb378075200 0x7fb37814b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3780cc190 0x7fb37814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb37815edc0 con 0x7fb3781517f0 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb37815eae0 con 0x7fb3780cc190 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb37dcc1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb37815f0a0 con 0x7fb378075200 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58590/0 (socket says 172.21.15.49:58590) 2024-05-09T08:55:57.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb376ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3780cc190 0x7fb37814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 -- 172.21.15.49:0/3176626267 learned_addr learned my addr 172.21.15.49:0/3176626267 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb3777fe640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb378075200 0x7fb37814b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 -- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb378075200 msgr2=0x7fb37814b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb378075200 0x7fb37814b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 -- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3780cc190 msgr2=0x7fb37814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3780cc190 0x7fb37814bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 -- 172.21.15.49:0/3176626267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3781f1780 con 0x7fb3781517f0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.851+0000 7fb377fff640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb370009eb0 tx=0x7fb370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.852+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370041030 con 0x7fb3781517f0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.852+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3700319c0 con 0x7fb3781517f0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.852+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781f1a60 con 0x7fb3781517f0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.852+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370009450 con 0x7fb3781517f0 2024-05-09T08:55:57.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.852+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb378158dc0 con 0x7fb3781517f0 2024-05-09T08:55:57.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.853+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb378072b70 con 0x7fb3781517f0 2024-05-09T08:55:57.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.854+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fb370044030 con 0x7fb3781517f0 2024-05-09T08:55:57.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.854+0000 7fb35ffff640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 0x7fb36406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:57.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.854+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7fb3700abc20 con 0x7fb3781517f0 2024-05-09T08:55:57.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.854+0000 7fb3777fe640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 0x7fb36406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:57.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.855+0000 7fb3777fe640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 0x7fb36406e260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb368009560 tx=0x7fb368009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:57.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:57.859+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb378072b70 con 0x7fb3781517f0 2024-05-09T08:55:58.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.090+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb378068c70 con 0x7fb3781517f0 2024-05-09T08:55:58.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.091+0000 7fb35ffff640 1 -- 172.21.15.49:0/3176626267 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5518 (secure 0 0 0) 0x7fb378068c70 con 0x7fb3781517f0 2024-05-09T08:55:58.090 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:58.091 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4285,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T08:55:58.091 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 3 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 msgr2=0x7fb36406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 0x7fb36406e260 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb368009560 tx=0x7fb368009190 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 msgr2=0x7fb3781598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb370009eb0 tx=0x7fb370004a40 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 shutdown_connections 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb36406bda0 0x7fb36406e260 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb3781517f0 0x7fb3781598a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3780cc190 0x7fb37814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 --2- 172.21.15.49:0/3176626267 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb378075200 0x7fb37814b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 >> 172.21.15.49:0/3176626267 conn(0x7fb37806bd00 msgr2=0x7fb378072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 shutdown_connections 2024-05-09T08:55:58.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.093+0000 7fb37dcc1640 1 -- 172.21.15.49:0/3176626267 wait complete. 2024-05-09T08:55:58.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:55:58.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.208+0000 7f19b9a72640 1 Processor -- start 2024-05-09T08:55:58.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.208+0000 7f19b9a72640 1 -- start start 2024-05-09T08:55:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.208+0000 7f19b9a72640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f19b4075200 0x7f19b415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.208+0000 7f19b9a72640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19b40cc190 0x7f19b40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b9a72640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19b40ccb20 con 0x7f19b41517f0 2024-05-09T08:55:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b9a72640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f19b4151bd0 con 0x7f19b40cc190 2024-05-09T08:55:58.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b9a72640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f19b415e260 con 0x7f19b4075200 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19b40cc190 0x7f19b40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f19b40cc190 0x7f19b40cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58592/0 (socket says 172.21.15.49:58592) 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 -- 172.21.15.49:0/3672352257 learned_addr learned my addr 172.21.15.49:0/3672352257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b2ffd640 1 --2- 172.21.15.49:0/3672352257 >> v2:172.21.15.183:3300/0 conn(0x7f19b4075200 0x7f19b415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 -- 172.21.15.49:0/3672352257 >> v2:172.21.15.183:3300/0 conn(0x7f19b4075200 msgr2=0x7f19b415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 --2- 172.21.15.49:0/3672352257 >> v2:172.21.15.183:3300/0 conn(0x7f19b4075200 0x7f19b415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 -- 172.21.15.49:0/3672352257 >> v1:172.21.15.183:6789/0 conn(0x7f19b41517f0 legacy=0x7f19b40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.209+0000 7f19b3fff640 1 -- 172.21.15.49:0/3672352257 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b415e520 con 0x7f19b40cc190 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b3fff640 1 --2- 172.21.15.49:0/3672352257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b40cc5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19b40cdc20 tx=0x7f19ac02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ef0ad3c7a56193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b27fc640 1 -- 172.21.15.49:0/3672352257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac036070 con 0x7f19b40cc190 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b27fc640 1 -- 172.21.15.49:0/3672352257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ac031600 con 0x7f19b40cc190 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b27fc640 1 -- 172.21.15.49:0/3672352257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0319f0 con 0x7f19b40cc190 2024-05-09T08:55:58.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 -- 172.21.15.49:0/3672352257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 msgr2=0x7f19b40cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 --2- 172.21.15.49:0/3672352257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b40cc5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19b40cdc20 tx=0x7f19ac02a7c0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 -- 172.21.15.49:0/3672352257 shutdown_connections 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 --2- 172.21.15.49:0/3672352257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b40cc5e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 --2- 172.21.15.49:0/3672352257 >> v2:172.21.15.183:3300/0 conn(0x7f19b4075200 0x7f19b415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 -- 172.21.15.49:0/3672352257 >> 172.21.15.49:0/3672352257 conn(0x7f19b406bd00 msgr2=0x7f19b4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.210+0000 7f19b9a72640 1 -- 172.21.15.49:0/3672352257 shutdown_connections 2024-05-09T08:55:58.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.211+0000 7f19b9a72640 1 -- 172.21.15.49:0/3672352257 wait complete. 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.212+0000 7f19b9a72640 1 Processor -- start 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.212+0000 7f19b9a72640 1 -- start start 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19b4075200 0x7f19b41f1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19b41517f0 0x7f19b414a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f19b415edc0 con 0x7f19b40cc190 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19b415eae0 con 0x7f19b4075200 2024-05-09T08:55:58.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f19b415f0a0 con 0x7f19b41517f0 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58602/0 (socket says 172.21.15.49:58602) 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 -- 172.21.15.49:0/160857720 learned_addr learned my addr 172.21.15.49:0/160857720 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b37fe640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19b4075200 0x7f19b41f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b3fff640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19b41517f0 0x7f19b414a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 -- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19b41517f0 msgr2=0x7f19b414a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19b41517f0 0x7f19b414a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 -- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19b4075200 msgr2=0x7f19b41f1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19b4075200 0x7f19b41f1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 -- 172.21.15.49:0/160857720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b415df10 con 0x7f19b40cc190 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b2ffd640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d800 tx=0x7f19a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a800b6c0 con 0x7f19b40cc190 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a8019070 con 0x7f19b40cc190 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b40ca780 con 0x7f19b40cc190 2024-05-09T08:55:58.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.213+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a800ba80 con 0x7f19b40cc190 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.214+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b415d950 con 0x7f19b40cc190 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.214+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b4150b80 con 0x7f19b40cc190 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.215+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f19a800bc20 con 0x7f19b40cc190 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.216+0000 7f199bfff640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 0x7f19a006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.216+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f19a808ae50 con 0x7f19b40cc190 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.216+0000 7f19b37fe640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 0x7f19a006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.216+0000 7f19b37fe640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 0x7f19a006e260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19a40016e0 tx=0x7f19a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.220+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f19b4150b80 con 0x7f19b40cc190 2024-05-09T08:55:58.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.407+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19b4068560 con 0x7f19b40cc190 2024-05-09T08:55:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.407+0000 7f199bfff640 1 -- 172.21.15.49:0/160857720 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12885 (secure 0 0 0) 0x7f19a8059550 con 0x7f19b40cc190 2024-05-09T08:55:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:58.407 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:55:57.652757+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 msgr2=0x7f19a006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 0x7f19a006e260 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19a40016e0 tx=0x7f19a4009040 comp rx=0 tx=0).stop 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 msgr2=0x7f19b4149d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19a800d800 tx=0x7f19a800dcc0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 shutdown_connections 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f19a006bda0 0x7f19a006e260 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19b41517f0 0x7f19b414a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19b40cc190 0x7f19b4149d70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 --2- 172.21.15.49:0/160857720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19b4075200 0x7f19b41f1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.409+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 >> 172.21.15.49:0/160857720 conn(0x7f19b406bd00 msgr2=0x7f19b414d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.410+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 shutdown_connections 2024-05-09T08:55:58.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.410+0000 7f19b9a72640 1 -- 172.21.15.49:0/160857720 wait complete. 2024-05-09T08:55:58.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:55:58.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:55:58.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.527+0000 7f3680038640 1 Processor -- start 2024-05-09T08:55:58.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.527+0000 7f3680038640 1 -- start start 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f3680038640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3678072830 0x7f3678072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f3680038640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367806fd40 con 0x7f3678154d50 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f3680038640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367815b940 con 0x7f36780707d0 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f3680038640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3678070c40 con 0x7f3678072830 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f367ddd4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3678154d50 0x7f367806f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58444/0 (socket says 172.21.15.49:58444) 2024-05-09T08:55:58.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f367ddd4640 1 -- 172.21.15.49:0/2558828746 learned_addr learned my addr 172.21.15.49:0/2558828746 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.528+0000 7f367e5d5640 1 --2- 172.21.15.49:0/2558828746 >> v2:172.21.15.49:3300/0 conn(0x7f3678072830 0x7f3678072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367cdd2640 1 -- 172.21.15.49:0/2558828746 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342829873 0 0) 0x7f367806fd40 con 0x7f3678154d50 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367cdd2640 1 -- 172.21.15.49:0/2558828746 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3660003520 con 0x7f3678154d50 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367e5d5640 1 -- 172.21.15.49:0/2558828746 >> v1:172.21.15.183:6789/0 conn(0x7f36780707d0 legacy=0x7f3678072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367e5d5640 1 -- 172.21.15.49:0/2558828746 >> v1:172.21.15.49:6789/0 conn(0x7f3678154d50 legacy=0x7f367806f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367e5d5640 1 -- 172.21.15.49:0/2558828746 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367815bc20 con 0x7f3678072830 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.529+0000 7f367e5d5640 1 --2- 172.21.15.49:0/2558828746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678072830 0x7f3678072c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3668009d90 tx=0x7f366802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5413aaaa6152ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f367cdd2640 1 -- 172.21.15.49:0/2558828746 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668037070 con 0x7f3678072830 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f367cdd2640 1 -- 172.21.15.49:0/2558828746 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3668007740 con 0x7f3678072830 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f367cdd2640 1 -- 172.21.15.49:0/2558828746 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668007b10 con 0x7f3678072830 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 -- 172.21.15.49:0/2558828746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678072830 msgr2=0x7f3678072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 --2- 172.21.15.49:0/2558828746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678072830 0x7f3678072c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3668009d90 tx=0x7f366802a780 comp rx=0 tx=0).stop 2024-05-09T08:55:58.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 -- 172.21.15.49:0/2558828746 shutdown_connections 2024-05-09T08:55:58.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 --2- 172.21.15.49:0/2558828746 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3678072830 0x7f3678072c40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 -- 172.21.15.49:0/2558828746 >> 172.21.15.49:0/2558828746 conn(0x7f36780ce390 msgr2=0x7f36780ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 -- 172.21.15.49:0/2558828746 shutdown_connections 2024-05-09T08:55:58.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.530+0000 7f3680038640 1 -- 172.21.15.49:0/2558828746 wait complete. 2024-05-09T08:55:58.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.531+0000 7f3680038640 1 Processor -- start 2024-05-09T08:55:58.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 -- start start 2024-05-09T08:55:58.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3678072830 0x7f36781f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678154d50 0x7f367814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f367806b410 con 0x7f36780707d0 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36780c9110 con 0x7f3678154d50 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f3680038640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3678155130 con 0x7f3678072830 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f367ddd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f367ddd4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58618/0 (socket says 172.21.15.49:58618) 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f367ddd4640 1 -- 172.21.15.49:0/1614014995 learned_addr learned my addr 172.21.15.49:0/1614014995 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f367e5d5640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678154d50 0x7f367814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.532+0000 7f367d5d3640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3678072830 0x7f36781f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 -- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3678072830 msgr2=0x7f36781f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3678072830 0x7f36781f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 -- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678154d50 msgr2=0x7f367814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678154d50 0x7f367814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 -- 172.21.15.49:0/1614014995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36781f4980 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f367ddd4640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f367000c440 tx=0x7f367000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670017070 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3670005ce0 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367815bee0 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36700072b0 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.533+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367815c460 con 0x7f36780707d0 2024-05-09T08:55:58.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.534+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367804f660 con 0x7f36780707d0 2024-05-09T08:55:58.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.534+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f367001a070 con 0x7f36780707d0 2024-05-09T08:55:58.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.535+0000 7f36767fc640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 0x7f366406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.535+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f3670089d50 con 0x7f36780707d0 2024-05-09T08:55:58.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.535+0000 7f367d5d3640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 0x7f366406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.536+0000 7f367d5d3640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 0x7f366406e260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36781f4d80 tx=0x7f366c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.540+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3670058510 con 0x7f36780707d0 2024-05-09T08:55:58.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.729+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f367806b820 con 0x7f36780707d0 2024-05-09T08:55:58.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.730+0000 7f36767fc640 1 -- 172.21.15.49:0/1614014995 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f367005c810 con 0x7f36780707d0 2024-05-09T08:55:58.729 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 msgr2=0x7f366406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 0x7f366406e260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36781f4d80 tx=0x7f366c009040 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 msgr2=0x7f36781e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f367000c440 tx=0x7f367000c900 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 shutdown_connections 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f366406bda0 0x7f366406e260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3678154d50 0x7f367814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3678072830 0x7f36781f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 --2- 172.21.15.49:0/1614014995 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f36780707d0 0x7f36781e4bf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 >> 172.21.15.49:0/1614014995 conn(0x7f36780ce390 msgr2=0x7f367806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 shutdown_connections 2024-05-09T08:55:58.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.732+0000 7f3680038640 1 -- 172.21.15.49:0/1614014995 wait complete. 2024-05-09T08:55:58.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T08:55:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.846+0000 7fc73a929640 1 Processor -- start 2024-05-09T08:55:58.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.846+0000 7fc73a929640 1 -- start start 2024-05-09T08:55:58.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.847+0000 7fc73a929640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc734154c70 0x7fc73406b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.847+0000 7fc73a929640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc73415b470 0x7fc73415b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.847+0000 7fc73a929640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc734072d90 con 0x7fc7340c8d20 2024-05-09T08:55:58.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.847+0000 7fc73a929640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc734072ac0 con 0x7fc734154c70 2024-05-09T08:55:58.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.847+0000 7fc73a929640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc734070700 con 0x7fc73415b470 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc73415b470 0x7fc73415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc73415b470 0x7fc73415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:52828/0 (socket says 172.21.15.49:52828) 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 -- 172.21.15.49:0/3430650087 learned_addr learned my addr 172.21.15.49:0/3430650087 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc733fff640 1 --2- 172.21.15.49:0/3430650087 >> v2:172.21.15.49:3300/0 conn(0x7fc734154c70 0x7fc73406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 -- 172.21.15.49:0/3430650087 >> v2:172.21.15.49:3300/0 conn(0x7fc734154c70 msgr2=0x7fc73406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 --2- 172.21.15.49:0/3430650087 >> v2:172.21.15.49:3300/0 conn(0x7fc734154c70 0x7fc73406b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 -- 172.21.15.49:0/3430650087 >> v1:172.21.15.49:6789/0 conn(0x7fc7340c8d20 legacy=0x7fc73415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:58.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.848+0000 7fc738ec6640 1 -- 172.21.15.49:0/3430650087 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7340709e0 con 0x7fc73415b470 2024-05-09T08:55:58.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc738ec6640 1 --2- 172.21.15.49:0/3430650087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc73415b470 0x7fc73415b880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7341588a0 tx=0x7fc72c02a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58d8526460eac95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc732ffd640 1 -- 172.21.15.49:0/3430650087 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c036070 con 0x7fc73415b470 2024-05-09T08:55:58.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc732ffd640 1 -- 172.21.15.49:0/3430650087 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c031600 con 0x7fc73415b470 2024-05-09T08:55:58.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc732ffd640 1 -- 172.21.15.49:0/3430650087 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0319d0 con 0x7fc73415b470 2024-05-09T08:55:58.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 -- 172.21.15.49:0/3430650087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc73415b470 msgr2=0x7fc73415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 --2- 172.21.15.49:0/3430650087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc73415b470 0x7fc73415b880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7341588a0 tx=0x7fc72c02a780 comp rx=0 tx=0).stop 2024-05-09T08:55:58.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 -- 172.21.15.49:0/3430650087 shutdown_connections 2024-05-09T08:55:58.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 --2- 172.21.15.49:0/3430650087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc73415b470 0x7fc73415b880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 --2- 172.21.15.49:0/3430650087 >> v2:172.21.15.49:3300/0 conn(0x7fc734154c70 0x7fc73406b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 -- 172.21.15.49:0/3430650087 >> 172.21.15.49:0/3430650087 conn(0x7fc734074840 msgr2=0x7fc734074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:58.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 -- 172.21.15.49:0/3430650087 shutdown_connections 2024-05-09T08:55:58.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.849+0000 7fc73a929640 1 -- 172.21.15.49:0/3430650087 wait complete. 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.851+0000 7fc73a929640 1 Processor -- start 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.851+0000 7fc73a929640 1 -- start start 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 0x7fc7340c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 0x7fc7341f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc73415b470 0x7fc73414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc73406d140 con 0x7fc734154c70 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc73406fff0 con 0x7fc7340c8d20 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc73415c250 con 0x7fc73415b470 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 0x7fc7341f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc7337fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 0x7fc7341f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58626/0 (socket says 172.21.15.49:58626) 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc7337fe640 1 -- 172.21.15.49:0/2226546520 learned_addr learned my addr 172.21.15.49:0/2226546520 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:58.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 0x7fc7340c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc738ec6640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc73415b470 0x7fc73414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 -- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc73415b470 msgr2=0x7fc73414f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc73415b470 0x7fc73414f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 -- 172.21.15.49:0/2226546520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 msgr2=0x7fc7341f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 0x7fc7341f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 -- 172.21.15.49:0/2226546520 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc734156ef0 con 0x7fc7340c8d20 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc733fff640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 0x7fc7340c70b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d5f0 tx=0x7fc72400dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240125f0 con 0x7fc7340c8d20 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc724019070 con 0x7fc7340c8d20 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240129d0 con 0x7fc7340c8d20 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.852+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7341f4bb0 con 0x7fc7340c8d20 2024-05-09T08:55:58.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.853+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7340c9320 con 0x7fc7340c8d20 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.853+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc734068c70 con 0x7fc7340c8d20 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.854+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fc724012b70 con 0x7fc7340c8d20 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.854+0000 7fc730ff9640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 0x7fc72006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.855+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fc72408ac80 con 0x7fc7340c8d20 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.855+0000 7fc7337fe640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 0x7fc72006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:58.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.855+0000 7fc7337fe640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 0x7fc72006e290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7341f4dc0 tx=0x7fc728007ef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:58.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:58.859+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc734156ef0 con 0x7fc7340c8d20 2024-05-09T08:55:59.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.052+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc73406f570 con 0x7fc7340c8d20 2024-05-09T08:55:59.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.053+0000 7fc730ff9640 1 -- 172.21.15.49:0/2226546520 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fc724059400 con 0x7fc7340c8d20 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout:[ 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout: "poolnum": 1, 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout: "poolname": ".mgr" 2024-05-09T08:55:59.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:55:59.054 INFO:teuthology.orchestra.run.smithi049.stdout:] 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 msgr2=0x7fc72006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 0x7fc72006e290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7341f4dc0 tx=0x7fc728007ef0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 msgr2=0x7fc7340c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 0x7fc7340c70b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d5f0 tx=0x7fc72400dab0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 shutdown_connections 2024-05-09T08:55:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc72006bdd0 0x7fc72006e290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc73415b470 0x7fc73414f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc734154c70 0x7fc7341f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 --2- 172.21.15.49:0/2226546520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7340c8d20 0x7fc7340c70b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 >> 172.21.15.49:0/2226546520 conn(0x7fc734074840 msgr2=0x7fc7340c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:59.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 shutdown_connections 2024-05-09T08:55:59.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.055+0000 7fc73a929640 1 -- 172.21.15.49:0/2226546520 wait complete. 2024-05-09T08:55:59.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T08:55:59.066 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T08:55:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.175+0000 7f3eaaff9640 1 Processor -- start 2024-05-09T08:55:59.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.175+0000 7f3eaaff9640 1 -- start start 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3eaaff9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea4073580 0x7f3ea4072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3eaaff9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ea406ff70 0x7f3ea4072870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3eaaff9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea415b750 con 0x7f3ea41550a0 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3eaaff9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea40c8e90 con 0x7f3ea406ff70 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3eaaff9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea406b710 con 0x7f3ea4073580 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ea406ff70 0x7f3ea4072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ea406ff70 0x7f3ea4072870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58642/0 (socket says 172.21.15.49:58642) 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 -- 172.21.15.49:0/3986826577 learned_addr learned my addr 172.21.15.49:0/3986826577 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 -- 172.21.15.49:0/3986826577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea4073580 msgr2=0x7f3ea4072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 --2- 172.21.15.49:0/3986826577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea4073580 0x7f3ea4072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 -- 172.21.15.49:0/3986826577 >> v1:172.21.15.183:6789/0 conn(0x7f3ea41550a0 legacy=0x7f3ea406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:59.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.176+0000 7f3ea9596640 1 -- 172.21.15.49:0/3986826577 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea415ba10 con 0x7f3ea406ff70 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.177+0000 7f3ea9596640 1 --2- 172.21.15.49:0/3986826577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea406ff70 0x7f3ea4072870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009870 tx=0x7f3e9c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305b613ebd2dcc77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.177+0000 7f3e9b7fe640 1 -- 172.21.15.49:0/3986826577 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c037070 con 0x7f3ea406ff70 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.177+0000 7f3e9b7fe640 1 -- 172.21.15.49:0/3986826577 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c032440 con 0x7f3ea406ff70 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.177+0000 7f3e9b7fe640 1 -- 172.21.15.49:0/3986826577 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c032810 con 0x7f3ea406ff70 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 -- 172.21.15.49:0/3986826577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea406ff70 msgr2=0x7f3ea4072870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/3986826577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea406ff70 0x7f3ea4072870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009870 tx=0x7f3e9c02a2a0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 -- 172.21.15.49:0/3986826577 shutdown_connections 2024-05-09T08:55:59.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/3986826577 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea406ff70 0x7f3ea4072870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/3986826577 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea4073580 0x7f3ea4072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 -- 172.21.15.49:0/3986826577 >> 172.21.15.49:0/3986826577 conn(0x7f3ea40ce490 msgr2=0x7f3ea40ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 -- 172.21.15.49:0/3986826577 shutdown_connections 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.178+0000 7f3eaaff9640 1 -- 172.21.15.49:0/3986826577 wait complete. 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.179+0000 7f3eaaff9640 1 Processor -- start 2024-05-09T08:55:59.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.179+0000 7f3eaaff9640 1 -- start start 2024-05-09T08:55:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 0x7f3ea41e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 0x7f3ea415e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4070a00 con 0x7f3ea41550a0 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea406b3a0 con 0x7f3ea4073580 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3eaaff9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea4070b80 con 0x7f3ea406ff70 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58646/0 (socket says 172.21.15.49:58646) 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 -- 172.21.15.49:0/4210290282 learned_addr learned my addr 172.21.15.49:0/4210290282 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3e9bfff640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 0x7f3ea415e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea8d95640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 0x7f3ea41e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 -- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 msgr2=0x7f3ea41e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 0x7f3ea41e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 -- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 msgr2=0x7f3ea415e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 0x7f3ea415e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 -- 172.21.15.49:0/4210290282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea415e5e0 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3e9bfff640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 0x7f3ea415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea8d95640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 0x7f3ea41e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.180+0000 7f3ea9596640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c02a2e0 tx=0x7f3e9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.181+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c041070 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.181+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e9c03c950 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.181+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c037040 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.181+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea415ef90 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.181+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41e5d10 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.182+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea4068e50 con 0x7f3ea41550a0 2024-05-09T08:55:59.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.182+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f3e9c044070 con 0x7f3ea41550a0 2024-05-09T08:55:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.183+0000 7f3e997fa640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 0x7f3e9006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.183+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3e9c0ab8a0 con 0x7f3ea41550a0 2024-05-09T08:55:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.183+0000 7f3ea8d95640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 0x7f3e9006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.183+0000 7f3ea8d95640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 0x7f3e9006e260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007e90 tx=0x7f3ea00073a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.187+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ea4068e50 con 0x7f3ea41550a0 2024-05-09T08:55:59.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.369+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3ea4068560 con 0x7f3ea41550a0 2024-05-09T08:55:59.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.563+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e9c03a070 con 0x7f3ea41550a0 2024-05-09T08:55:59.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.666+0000 7f3e997fa640 1 -- 172.21.15.49:0/4210290282 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f3ea415e5e0 con 0x7f3ea41550a0 2024-05-09T08:55:59.665 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' created 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 msgr2=0x7f3e9006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 0x7f3e9006e260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0007e90 tx=0x7f3ea00073a0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 msgr2=0x7f3ea41e6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c02a2e0 tx=0x7f3e9c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 shutdown_connections 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e9006bda0 0x7f3e9006e260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ea41550a0 0x7f3ea41e6ad0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ea4073580 0x7f3ea415e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 --2- 172.21.15.49:0/4210290282 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ea406ff70 0x7f3ea41e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 >> 172.21.15.49:0/4210290282 conn(0x7f3ea40ce490 msgr2=0x7f3ea40c78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 shutdown_connections 2024-05-09T08:55:59.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.669+0000 7f3eaaff9640 1 -- 172.21.15.49:0/4210290282 wait complete. 2024-05-09T08:55:59.683 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T08:55:59.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.795+0000 7fb5c2fe9640 1 Processor -- start 2024-05-09T08:55:59.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.795+0000 7fb5c2fe9640 1 -- start start 2024-05-09T08:55:59.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c2fe9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb5bc1550f0 0x7fb5bc15b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c2fe9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5bc15e810 0x7fb5bc15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c2fe9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5bc0b36b0 con 0x7fb5bc0d10a0 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c2fe9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc1554d0 con 0x7fb5bc15e810 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c2fe9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb5bc15f180 con 0x7fb5bc1550f0 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c1586640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5bc15e810 0x7fb5bc15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5bbfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb5bc0d10a0 0x7fb5bc072230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58454/0 (socket says 172.21.15.49:58454) 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5bbfff640 1 -- 172.21.15.49:0/770731589 learned_addr learned my addr 172.21.15.49:0/770731589 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c0d85640 1 --2- 172.21.15.49:0/770731589 >> v2:172.21.15.183:3300/0 conn(0x7fb5bc1550f0 0x7fb5bc15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c1586640 1 -- 172.21.15.49:0/770731589 >> v2:172.21.15.183:3300/0 conn(0x7fb5bc1550f0 msgr2=0x7fb5bc15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c1586640 1 --2- 172.21.15.49:0/770731589 >> v2:172.21.15.183:3300/0 conn(0x7fb5bc1550f0 0x7fb5bc15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c1586640 1 -- 172.21.15.49:0/770731589 >> v1:172.21.15.49:6789/0 conn(0x7fb5bc0d10a0 legacy=0x7fb5bc072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5bb7fe640 1 -- 172.21.15.49:0/770731589 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618045484 0 0) 0x7fb5bc0b36b0 con 0x7fb5bc0d10a0 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.796+0000 7fb5c1586640 1 -- 172.21.15.49:0/770731589 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc0cbff0 con 0x7fb5bc15e810 2024-05-09T08:55:59.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.797+0000 7fb5c1586640 1 --2- 172.21.15.49:0/770731589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc15ec40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0c5660 tx=0x7fb5ac02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abe8afe7c35289f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.797+0000 7fb5bb7fe640 1 -- 172.21.15.49:0/770731589 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac007740 con 0x7fb5bc15e810 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.797+0000 7fb5bb7fe640 1 -- 172.21.15.49:0/770731589 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac0078e0 con 0x7fb5bc15e810 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.797+0000 7fb5bb7fe640 1 -- 172.21.15.49:0/770731589 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac007cb0 con 0x7fb5bc15e810 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/770731589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 msgr2=0x7fb5bc15ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/770731589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc15ec40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0c5660 tx=0x7fb5ac02a710 comp rx=0 tx=0).stop 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/770731589 shutdown_connections 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/770731589 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc15ec40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/770731589 >> v2:172.21.15.183:3300/0 conn(0x7fb5bc1550f0 0x7fb5bc15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/770731589 >> 172.21.15.49:0/770731589 conn(0x7fb5bc071a00 msgr2=0x7fb5bc067760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/770731589 shutdown_connections 2024-05-09T08:55:59.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.798+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/770731589 wait complete. 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.799+0000 7fb5c2fe9640 1 Processor -- start 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.799+0000 7fb5c2fe9640 1 -- start start 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5bc0d10a0 0x7fb5bc1f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5bc1550f0 0x7fb5bc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0cc890 con 0x7fb5bc15e810 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc0cc5b0 con 0x7fb5bc0d10a0 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c2fe9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb5bc0ccb70 con 0x7fb5bc1550f0 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58670/0 (socket says 172.21.15.49:58670) 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 -- 172.21.15.49:0/2311082615 learned_addr learned my addr 172.21.15.49:0/2311082615 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5bbfff640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5bc1550f0 0x7fb5bc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 -- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5bc1550f0 msgr2=0x7fb5bc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5bc1550f0 0x7fb5bc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.800+0000 7fb5c1586640 1 -- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5bc0d10a0 msgr2=0x7fb5bc1f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5c1586640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5bc0d10a0 0x7fb5bc1f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5c1586640 1 -- 172.21.15.49:0/2311082615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc14ec50 con 0x7fb5bc15e810 2024-05-09T08:55:59.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5c1586640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac02ad00 tx=0x7fb5ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac03f070 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac031bc0 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac044460 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc1f1890 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.801+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1f15b0 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.802+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc0c7560 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.802+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5ac007790 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.803+0000 7fb5b97fa640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 0x7fb59c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.803+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fb5ac0ac450 con 0x7fb5bc15e810 2024-05-09T08:55:59.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.803+0000 7fb5c0d85640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 0x7fb59c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:55:59.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.803+0000 7fb5c0d85640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 0x7fb59c06e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40098f0 tx=0x7fb5b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:55:59.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.807+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5bc0c7560 con 0x7fb5bc15e810 2024-05-09T08:55:59.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:55:59.993+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb5bc068560 con 0x7fb5bc15e810 2024-05-09T08:56:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.672+0000 7fb5b97fa640 1 -- 172.21.15.49:0/2311082615 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fb5bc14ec50 con 0x7fb5bc15e810 2024-05-09T08:56:00.671 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' created 2024-05-09T08:56:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 msgr2=0x7fb59c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 0x7fb59c06e2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40098f0 tx=0x7fb5b4008040 comp rx=0 tx=0).stop 2024-05-09T08:56:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 msgr2=0x7fb5bc150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:00.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac02ad00 tx=0x7fb5ac004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 shutdown_connections 2024-05-09T08:56:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb59c06be30 0x7fb59c06e2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5bc15e810 0x7fb5bc150ea0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5bc1550f0 0x7fb5bc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 --2- 172.21.15.49:0/2311082615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5bc0d10a0 0x7fb5bc1f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 >> 172.21.15.49:0/2311082615 conn(0x7fb5bc071a00 msgr2=0x7fb5bc0ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 shutdown_connections 2024-05-09T08:56:00.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.675+0000 7fb5c2fe9640 1 -- 172.21.15.49:0/2311082615 wait complete. 2024-05-09T08:56:00.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T08:56:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.811+0000 7f7a3ac4c640 1 Processor -- start 2024-05-09T08:56:00.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.811+0000 7f7a3ac4c640 1 -- start start 2024-05-09T08:56:00.812 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a3ac4c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 0x7f7a340c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a3ac4c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a340c2730 0x7f7a340c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a3ac4c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a3415f160 con 0x7f7a34155180 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a3ac4c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7a340c7b90 con 0x7f7a340c2730 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a3ac4c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a340c78b0 con 0x7f7a340c8f90 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a340c2730 0x7f7a340c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a340c2730 0x7f7a340c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58672/0 (socket says 172.21.15.49:58672) 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 -- 172.21.15.49:0/2096929219 learned_addr learned my addr 172.21.15.49:0/2096929219 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a2bfff640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 0x7f7a340c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 -- 172.21.15.49:0/2096929219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 msgr2=0x7f7a340c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 0x7f7a340c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.812+0000 7f7a391e9640 1 -- 172.21.15.49:0/2096929219 >> v1:172.21.15.183:6789/0 conn(0x7f7a34155180 legacy=0x7f7a3406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a391e9640 1 -- 172.21.15.49:0/2096929219 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3415e3a0 con 0x7f7a340c2730 2024-05-09T08:56:00.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a2bfff640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 0x7f7a340c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a391e9640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c2730 0x7f7a340c2bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009e50 tx=0x7f7a2c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1259949ea4f265ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a2b7fe640 1 -- 172.21.15.49:0/2096929219 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c036070 con 0x7f7a340c2730 2024-05-09T08:56:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a2b7fe640 1 -- 172.21.15.49:0/2096929219 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a2c031600 con 0x7f7a340c2730 2024-05-09T08:56:00.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.813+0000 7f7a2b7fe640 1 -- 172.21.15.49:0/2096929219 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0319d0 con 0x7f7a340c2730 2024-05-09T08:56:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/2096929219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c2730 msgr2=0x7f7a340c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:00.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c2730 0x7f7a340c2bc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009e50 tx=0x7f7a2c02a630 comp rx=0 tx=0).stop 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/2096929219 shutdown_connections 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c2730 0x7f7a340c2bc0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/2096929219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c8f90 0x7f7a340c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/2096929219 >> 172.21.15.49:0/2096929219 conn(0x7f7a340c9650 msgr2=0x7f7a340c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/2096929219 shutdown_connections 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.814+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/2096929219 wait complete. 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.815+0000 7f7a3ac4c640 1 Processor -- start 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.815+0000 7f7a3ac4c640 1 -- start start 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c2730 0x7f7a340b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a34155180 0x7f7a340b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3406b910 con 0x7f7a340c8f90 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3415e960 con 0x7f7a34155180 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a3ac4c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a340c7550 con 0x7f7a340c2730 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a2bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58680/0 (socket says 172.21.15.49:58680) 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a2bfff640 1 -- 172.21.15.49:0/3359093674 learned_addr learned my addr 172.21.15.49:0/3359093674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a391e9640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a34155180 0x7f7a340b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a389e8640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c2730 0x7f7a340b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.816+0000 7f7a2bfff640 1 -- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c2730 msgr2=0x7f7a340b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a2bfff640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c2730 0x7f7a340b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a2bfff640 1 -- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a34155180 msgr2=0x7f7a340b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a2bfff640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a34155180 0x7f7a340b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:00.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a2bfff640 1 -- 172.21.15.49:0/3359093674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a340b0e10 con 0x7f7a340c8f90 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a2bfff640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c7f0 tx=0x7f7a2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24015070 con 0x7f7a340c8f90 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a24007a30 con 0x7f7a340c8f90 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a340b10f0 con 0x7f7a340c8f90 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24007e00 con 0x7f7a340c8f90 2024-05-09T08:56:00.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.817+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a340b16b0 con 0x7f7a340c8f90 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.818+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3404f660 con 0x7f7a340c8f90 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.819+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a24018070 con 0x7f7a340c8f90 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.819+0000 7f7a297fa640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 0x7f7a2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.819+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f7a2408ad40 con 0x7f7a340c8f90 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.819+0000 7f7a389e8640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 0x7f7a2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:00.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.820+0000 7f7a389e8640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 0x7f7a2006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:00.824+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a24059100 con 0x7f7a340c8f90 2024-05-09T08:56:01.046 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.047+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7a340c2200 con 0x7f7a340c8f90 2024-05-09T08:56:01.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.673+0000 7f7a297fa640 1 -- 172.21.15.49:0/3359093674 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7a2405d400 con 0x7f7a340c8f90 2024-05-09T08:56:01.673 INFO:teuthology.orchestra.run.smithi049.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T08:56:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 msgr2=0x7f7a2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 0x7f7a2006e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a30006fd0 tx=0x7f7a30009060 comp rx=0 tx=0).stop 2024-05-09T08:56:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 msgr2=0x7f7a340b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400c7f0 tx=0x7f7a2400cbf0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 shutdown_connections 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a2006be30 0x7f7a2006e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a34155180 0x7f7a340b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a340c8f90 0x7f7a340b1eb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 --2- 172.21.15.49:0/3359093674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a340c2730 0x7f7a340b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.678+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 >> 172.21.15.49:0/3359093674 conn(0x7f7a340c9650 msgr2=0x7f7a340cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.679+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 shutdown_connections 2024-05-09T08:56:01.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.679+0000 7f7a3ac4c640 1 -- 172.21.15.49:0/3359093674 wait complete. 2024-05-09T08:56:01.708 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:01.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.816+0000 7fc0cc41c640 1 Processor -- start 2024-05-09T08:56:01.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.816+0000 7fc0cc41c640 1 -- start start 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0cc41c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc0c40c8f70 0x7fc0c40c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0cc41c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c40ca590 con 0x7fc0c4153040 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0cc41c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c40c9980 con 0x7fc0c40c5fb0 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0cc41c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c40c9c60 con 0x7fc0c40c8f70 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0c99b7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc0c40c8f70 0x7fc0c40c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0ca1b8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc0c4153040 0x7fc0c40ce4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58464/0 (socket says 172.21.15.49:58464) 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.817+0000 7fc0ca1b8640 1 -- 172.21.15.49:0/3096278598 learned_addr learned my addr 172.21.15.49:0/3096278598 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c99b7640 1 -- 172.21.15.49:0/3096278598 >> v1:172.21.15.183:6789/0 conn(0x7fc0c40c5fb0 legacy=0x7fc0c4158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c99b7640 1 -- 172.21.15.49:0/3096278598 >> v1:172.21.15.49:6789/0 conn(0x7fc0c4153040 legacy=0x7fc0c40ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c99b7640 1 -- 172.21.15.49:0/3096278598 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c40c9f40 con 0x7fc0c40c8f70 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c99b7640 1 --2- 172.21.15.49:0/3096278598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c40c5a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40097a0 tx=0x7fc0b402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae9ca0149f647ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c91b6640 1 -- 172.21.15.49:0/3096278598 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4037070 con 0x7fc0c40c8f70 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c91b6640 1 -- 172.21.15.49:0/3096278598 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b4032640 con 0x7fc0c40c8f70 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.818+0000 7fc0c91b6640 1 -- 172.21.15.49:0/3096278598 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4032a10 con 0x7fc0c40c8f70 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 -- 172.21.15.49:0/3096278598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 msgr2=0x7fc0c40c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/3096278598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c40c5a70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40097a0 tx=0x7fc0b402a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 -- 172.21.15.49:0/3096278598 shutdown_connections 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/3096278598 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c40c5a70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 -- 172.21.15.49:0/3096278598 >> 172.21.15.49:0/3096278598 conn(0x7fc0c4067d90 msgr2=0x7fc0c4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:01.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 -- 172.21.15.49:0/3096278598 shutdown_connections 2024-05-09T08:56:01.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.819+0000 7fc0cc41c640 1 -- 172.21.15.49:0/3096278598 wait complete. 2024-05-09T08:56:01.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.820+0000 7fc0cc41c640 1 Processor -- start 2024-05-09T08:56:01.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.820+0000 7fc0cc41c640 1 -- start start 2024-05-09T08:56:01.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0c40c5fb0 0x7fc0c4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:01.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:01.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0c4153040 0x7fc0c40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4159540 con 0x7fc0c40c8f70 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c40cebd0 con 0x7fc0c4153040 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0cc41c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c41596c0 con 0x7fc0c40c5fb0 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58710/0 (socket says 172.21.15.49:58710) 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 -- 172.21.15.49:0/4130418306 learned_addr learned my addr 172.21.15.49:0/4130418306 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0ca9b9640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0c4153040 0x7fc0c40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0ca1b8640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0c40c5fb0 0x7fc0c4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 -- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0c40c5fb0 msgr2=0x7fc0c4159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0c40c5fb0 0x7fc0c4159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 -- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0c4153040 msgr2=0x7fc0c40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0c4153040 0x7fc0c40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:01.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.821+0000 7fc0c99b7640 1 -- 172.21.15.49:0/4130418306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c415bb00 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0c99b7640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4035040 tx=0x7fc0b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4041030 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b4030750 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c415bde0 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4037040 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.822+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c415cc80 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.823+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c4068560 con 0x7fc0c40c8f70 2024-05-09T08:56:01.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.823+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0b4044070 con 0x7fc0c40c8f70 2024-05-09T08:56:01.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.824+0000 7fc0baffd640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 0x7fc0a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:01.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.824+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fc0b40ab2d0 con 0x7fc0c40c8f70 2024-05-09T08:56:01.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.824+0000 7fc0ca1b8640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 0x7fc0a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:01.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.824+0000 7fc0ca1b8640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 0x7fc0a806e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005880 tx=0x7fc0c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:01.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:01.828+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0b4079bf0 con 0x7fc0c40c8f70 2024-05-09T08:56:02.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.016+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0c415c300 con 0x7fc0c40c8f70 2024-05-09T08:56:02.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.017+0000 7fc0baffd640 1 -- 172.21.15.49:0/4130418306 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16606 (secure 0 0 0) 0x7fc0b40794f0 con 0x7fc0c40c8f70 2024-05-09T08:56:02.017 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:02.017 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:01.048944+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 msgr2=0x7fc0a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 0x7fc0a806e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0005880 tx=0x7fc0c0009040 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 msgr2=0x7fc0c41e5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4035040 tx=0x7fc0b40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 shutdown_connections 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc0a806be30 0x7fc0a806e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0c4153040 0x7fc0c40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc0c40c8f70 0x7fc0c41e5880 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 --2- 172.21.15.49:0/4130418306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc0c40c5fb0 0x7fc0c4159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 >> 172.21.15.49:0/4130418306 conn(0x7fc0c4067d90 msgr2=0x7fc0c40cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:02.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 shutdown_connections 2024-05-09T08:56:02.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.019+0000 7fc0cc41c640 1 -- 172.21.15.49:0/4130418306 wait complete. 2024-05-09T08:56:02.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:02.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.135+0000 7ff9d2f3b640 1 Processor -- start 2024-05-09T08:56:02.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.135+0000 7ff9d2f3b640 1 -- start start 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9d2f3b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff9cc0d0500 0x7ff9cc0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9d2f3b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9cc0cb520 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9d2f3b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9cc0cb800 con 0x7ff9cc0d08e0 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9d2f3b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff9cc15e400 con 0x7ff9cc0d0500 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cbfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff9cc0c57f0 0x7ff9cc15ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58466/0 (socket says 172.21.15.49:58466) 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cbfff640 1 -- 172.21.15.49:0/2629598966 learned_addr learned my addr 172.21.15.49:0/2629598966 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/2629598966 >> v2:172.21.15.183:3300/0 conn(0x7ff9cc0d0500 0x7ff9cc0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 427485974 0 0) 0x7ff9cc0cb520 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9bc003520 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1762575978 0 0) 0x7ff9bc003520 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 >> v2:172.21.15.183:3300/0 conn(0x7ff9cc0d0500 msgr2=0x7ff9cc0b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 --2- 172.21.15.49:0/2629598966 >> v2:172.21.15.183:3300/0 conn(0x7ff9cc0d0500 0x7ff9cc0b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 >> v1:172.21.15.183:6789/0 conn(0x7ff9cc0d08e0 legacy=0x7ff9cc165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc15e6e0 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7ff9b8001670 con 0x7ff9cc0c57f0 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 >> v1:172.21.15.49:6789/0 conn(0x7ff9cc0c57f0 legacy=0x7ff9cc15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9bc0039b0 0x7ff9bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9bc0051c0 0x7ff9bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc15e400 con 0x7ff9bc004800 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc00c160 con 0x7ff9bc0039b0 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.136+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc00c360 con 0x7ff9bc0051c0 2024-05-09T08:56:02.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9bc0039b0 0x7ff9bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 -- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9bc0051c0 msgr2=0x7ff9bc00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9bc0051c0 0x7ff9bc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 -- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9bc0039b0 msgr2=0x7ff9bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9bc0039b0 0x7ff9bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9d14d8640 1 -- 172.21.15.49:0/2629598966 reap_dead start 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9cbfff640 1 -- 172.21.15.49:0/2629598966 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc003520 con 0x7ff9bc004800 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.137+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9bc0039b0 0x7ff9bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9bc004c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c240 tx=0x7ff9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d218b861e182da53 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8007d10 con 0x7ff9bc004800 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b8005ad0 con 0x7ff9bc004800 2024-05-09T08:56:02.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9cb7fe640 1 -- 172.21.15.49:0/2629598966 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8007070 con 0x7ff9bc004800 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 msgr2=0x7ff9bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9bc004c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c240 tx=0x7ff9bc005830 comp rx=0 tx=0).stop 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/2629598966 shutdown_connections 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/2629598966 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9bc004c80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/2629598966 >> 172.21.15.49:0/2629598966 conn(0x7ff9cc067940 msgr2=0x7ff9cc072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/2629598966 shutdown_connections 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.138+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/2629598966 wait complete. 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.139+0000 7ff9d2f3b640 1 Processor -- start 2024-05-09T08:56:02.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 -- start start 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9cc1f3fc0 0x7ff9cc0b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9cc1563b0 0x7ff9cc156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0cca40 con 0x7ff9bc004800 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc15bbe0 con 0x7ff9cc1f3fc0 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d2f3b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc0c5d40 con 0x7ff9cc1563b0 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d0cd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d0cd7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58734/0 (socket says 172.21.15.49:58734) 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.140+0000 7ff9d0cd7640 1 -- 172.21.15.49:0/1988000650 learned_addr learned my addr 172.21.15.49:0/1988000650 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 -- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9cc1563b0 msgr2=0x7ff9cc156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9cc1563b0 0x7ff9cc156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 -- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9cc1f3fc0 msgr2=0x7ff9cc0b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9cc1f3fc0 0x7ff9cc0b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 -- 172.21.15.49:0/1988000650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc156d70 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d0cd7640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011510 tx=0x7ff9c00119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0015870 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c0015a10 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc0b6680 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0015e00 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.141+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1f0e40 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.142+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc068560 con 0x7ff9bc004800 2024-05-09T08:56:02.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.143+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9c001d400 con 0x7ff9bc004800 2024-05-09T08:56:02.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.143+0000 7ff9c97fa640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 0x7ff9b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.143+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7ff9c008b6b0 con 0x7ff9bc004800 2024-05-09T08:56:02.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.143+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 0x7ff9b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.144+0000 7ff9cbfff640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 0x7ff9b006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8033000 tx=0x7ff9b8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.147+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9cc156d70 con 0x7ff9bc004800 2024-05-09T08:56:02.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.331+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9cc0b3000 con 0x7ff9bc004800 2024-05-09T08:56:02.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.332+0000 7ff9c97fa640 1 -- 172.21.15.49:0/1988000650 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16606 (secure 0 0 0) 0x7ff9c0019030 con 0x7ff9bc004800 2024-05-09T08:56:02.332 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:02.332 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":19,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:01.048944+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 msgr2=0x7ff9b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 0x7ff9b006e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8033000 tx=0x7ff9b8009030 comp rx=0 tx=0).stop 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 msgr2=0x7ff9cc157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0011510 tx=0x7ff9c00119d0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 shutdown_connections 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff9b006be30 0x7ff9b006e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9cc1563b0 0x7ff9cc156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9cc1f3fc0 0x7ff9cc0b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 --2- 172.21.15.49:0/1988000650 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9bc004800 0x7ff9cc157e30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 >> 172.21.15.49:0/1988000650 conn(0x7ff9cc067940 msgr2=0x7ff9cc0cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:02.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 shutdown_connections 2024-05-09T08:56:02.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.334+0000 7ff9d2f3b640 1 -- 172.21.15.49:0/1988000650 wait complete. 2024-05-09T08:56:02.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T08:56:02.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.452+0000 7fe5b1eb3640 1 Processor -- start 2024-05-09T08:56:02.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.452+0000 7fe5b1eb3640 1 -- start start 2024-05-09T08:56:02.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5b1eb3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac15e310 0x7fe5ac0ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5b1eb3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5ac0c9c80 0x7fe5ac1656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5b1eb3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5ac1553f0 con 0x7fe5ac155010 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5b1eb3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe5ac0cad60 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5b1eb3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac073240 con 0x7fe5ac15e310 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5abfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe5ac0c9c80 0x7fe5ac1656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5ab7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe5ac155010 0x7fe5ac0caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:58476/0 (socket says 172.21.15.49:58476) 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5ab7fe640 1 -- 172.21.15.49:0/2614323412 learned_addr learned my addr 172.21.15.49:0/2614323412 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5aaffd640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac15e310 0x7fe5ac0ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5abfff640 1 -- 172.21.15.49:0/2614323412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac15e310 msgr2=0x7fe5ac0ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5abfff640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac15e310 0x7fe5ac0ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5abfff640 1 -- 172.21.15.49:0/2614323412 >> v1:172.21.15.49:6789/0 conn(0x7fe5ac155010 legacy=0x7fe5ac0caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5aa7fc640 1 -- 172.21.15.49:0/2614323412 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606003945 0 0) 0x7fe5ac1553f0 con 0x7fe5ac155010 2024-05-09T08:56:02.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.453+0000 7fe5abfff640 1 -- 172.21.15.49:0/2614323412 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac0729a0 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.454+0000 7fe5abfff640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac1656a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009d90 tx=0x7fe5a402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d415159298259de1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.454+0000 7fe5aa7fc640 1 -- 172.21.15.49:0/2614323412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4007740 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.454+0000 7fe5aa7fc640 1 -- 172.21.15.49:0/2614323412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a40078e0 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.454+0000 7fe5aa7fc640 1 -- 172.21.15.49:0/2614323412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4007cb0 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/2614323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0c9c80 msgr2=0x7fe5ac1656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac1656a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009d90 tx=0x7fe5a402a660 comp rx=0 tx=0).stop 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/2614323412 shutdown_connections 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac1656a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/2614323412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac15e310 0x7fe5ac0ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/2614323412 >> 172.21.15.49:0/2614323412 conn(0x7fe5ac067d90 msgr2=0x7fe5ac074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/2614323412 shutdown_connections 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.455+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/2614323412 wait complete. 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.456+0000 7fe5b1eb3640 1 Processor -- start 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.456+0000 7fe5b1eb3640 1 -- start start 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac155010 0x7fe5ac0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac0ca300 con 0x7fe5ac15e310 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac072f60 con 0x7fe5ac0c9c80 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5b1eb3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ac073760 con 0x7fe5ac155010 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:58752/0 (socket says 172.21.15.49:58752) 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 -- 172.21.15.49:0/1100900327 learned_addr learned my addr 172.21.15.49:0/1100900327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 -- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac155010 msgr2=0x7fe5ac0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac155010 0x7fe5ac0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 -- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5ac0c9c80 msgr2=0x7fe5ac156930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac156930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.457+0000 7fe5abfff640 1 -- 172.21.15.49:0/1100900327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac0cc8f0 con 0x7fe5ac15e310 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe5abfff640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009eb0 tx=0x7fe5a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a403f070 con 0x7fe5ac15e310 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a4031c40 con 0x7fe5ac15e310 2024-05-09T08:56:02.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac0c5a80 con 0x7fe5ac15e310 2024-05-09T08:56:02.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4009450 con 0x7fe5ac15e310 2024-05-09T08:56:02.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.458+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac0c6020 con 0x7fe5ac15e310 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.459+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac068560 con 0x7fe5ac15e310 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.459+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5a4003990 con 0x7fe5ac15e310 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.460+0000 7fe593fff640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 0x7fe59806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.460+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fe5a40abcd0 con 0x7fe5ac15e310 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.461+0000 7fe5ab7fe640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 0x7fe59806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:02.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.461+0000 7fe5ab7fe640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 0x7fe59806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:02.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.464+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe5ac068560 con 0x7fe5ac15e310 2024-05-09T08:56:02.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:02.690+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe5ac04f660 con 0x7fe5ac15e310 2024-05-09T08:56:03.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.691+0000 7fe593fff640 1 -- 172.21.15.49:0/1100900327 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fe5a407a680 con 0x7fe5ac15e310 2024-05-09T08:56:03.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.693+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 msgr2=0x7fe59806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.693+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 0x7fe59806e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe59c007fd0 tx=0x7fe59c007960 comp rx=0 tx=0).stop 2024-05-09T08:56:03.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.693+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 msgr2=0x7fe5ac0c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.693+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009eb0 tx=0x7fe5a4004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:03.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 shutdown_connections 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe59806be30 0x7fe59806e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe5ac15e310 0x7fe5ac0c6cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5ac155010 0x7fe5ac0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 --2- 172.21.15.49:0/1100900327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5ac0c9c80 0x7fe5ac156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 >> 172.21.15.49:0/1100900327 conn(0x7fe5ac067d90 msgr2=0x7fe5ac074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 shutdown_connections 2024-05-09T08:56:03.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.694+0000 7fe5b1eb3640 1 -- 172.21.15.49:0/1100900327 wait complete. 2024-05-09T08:56:03.706 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-05-09T08:56:03.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-09T08:56:03.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.820+0000 7ff86e110640 1 Processor -- start 2024-05-09T08:56:03.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.820+0000 7ff86e110640 1 -- start start 2024-05-09T08:56:03.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff86e110640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8680c8f90 0x7ff8680c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff86e110640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8680c2730 0x7ff8680c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff86e110640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff86815f160 con 0x7ff868155180 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff86e110640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680c7b90 con 0x7ff8680c2730 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff86e110640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680c78b0 con 0x7ff8680c8f90 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff867fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff8680c2730 0x7ff8680c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff8677fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff868155180 0x7ff86806b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45866/0 (socket says 172.21.15.49:45866) 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff8677fe640 1 -- 172.21.15.49:0/1022452812 learned_addr learned my addr 172.21.15.49:0/1022452812 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff866ffd640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8680c8f90 0x7ff8680c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff867fff640 1 -- 172.21.15.49:0/1022452812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8680c8f90 msgr2=0x7ff8680c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:03.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff867fff640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8680c8f90 0x7ff8680c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff867fff640 1 -- 172.21.15.49:0/1022452812 >> v1:172.21.15.49:6789/0 conn(0x7ff868155180 legacy=0x7ff86806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff8667fc640 1 -- 172.21.15.49:0/1022452812 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3475023818 0 0) 0x7ff86815f160 con 0x7ff868155180 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.821+0000 7ff867fff640 1 -- 172.21.15.49:0/1022452812 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86815e3a0 con 0x7ff8680c2730 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.822+0000 7ff867fff640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8680c2bc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff860009d90 tx=0x7ff86002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87257b803f0a892b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.822+0000 7ff8667fc640 1 -- 172.21.15.49:0/1022452812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860031600 con 0x7ff8680c2730 2024-05-09T08:56:03.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.822+0000 7ff8667fc640 1 -- 172.21.15.49:0/1022452812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8600317a0 con 0x7ff8680c2730 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.822+0000 7ff8667fc640 1 -- 172.21.15.49:0/1022452812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860031b70 con 0x7ff8680c2730 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 -- 172.21.15.49:0/1022452812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 msgr2=0x7ff8680c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8680c2bc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff860009d90 tx=0x7ff86002a5d0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 -- 172.21.15.49:0/1022452812 shutdown_connections 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8680c2bc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 --2- 172.21.15.49:0/1022452812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff8680c8f90 0x7ff8680c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 -- 172.21.15.49:0/1022452812 >> 172.21.15.49:0/1022452812 conn(0x7ff8680c9650 msgr2=0x7ff8680c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 -- 172.21.15.49:0/1022452812 shutdown_connections 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.823+0000 7ff86e110640 1 -- 172.21.15.49:0/1022452812 wait complete. 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.824+0000 7ff86e110640 1 Processor -- start 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.824+0000 7ff86e110640 1 -- start start 2024-05-09T08:56:03.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8680c8f90 0x7ff868073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff868155180 0x7ff86814e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff86806b910 con 0x7ff8680c2730 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff86815e960 con 0x7ff8680c8f90 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff86e110640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680c7550 con 0x7ff868155180 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47818/0 (socket says 172.21.15.49:47818) 2024-05-09T08:56:03.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 -- 172.21.15.49:0/2941425192 learned_addr learned my addr 172.21.15.49:0/2941425192 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff866ffd640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8680c8f90 0x7ff868073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff867fff640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff868155180 0x7ff86814e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 -- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff868155180 msgr2=0x7ff86814e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff868155180 0x7ff86814e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 -- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8680c8f90 msgr2=0x7ff868073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8680c8f90 0x7ff868073110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.825+0000 7ff8677fe640 1 -- 172.21.15.49:0/2941425192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681f3e40 con 0x7ff8680c2730 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff8677fe640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff85800c460 tx=0x7ff85800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858018070 con 0x7ff8680c2730 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff858007500 con 0x7ff8680c2730 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681f4120 con 0x7ff8680c2730 2024-05-09T08:56:03.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858004a20 con 0x7ff8680c2730 2024-05-09T08:56:03.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.826+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681f46e0 con 0x7ff8680c2730 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.827+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86804f660 con 0x7ff8680c2730 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.827+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff85801b070 con 0x7ff8680c2730 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.828+0000 7ff84ffff640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 0x7ff85406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.828+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff85808a0c0 con 0x7ff8680c2730 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.828+0000 7ff866ffd640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 0x7ff85406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.828+0000 7ff866ffd640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 0x7ff85406e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff86815bca0 tx=0x7ff85c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:03.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:03.832+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff858058a70 con 0x7ff8680c2730 2024-05-09T08:56:04.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.053+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff8680c7d70 con 0x7ff8680c2730 2024-05-09T08:56:04.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.695+0000 7ff84ffff640 1 -- 172.21.15.49:0/2941425192 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7ff858058370 con 0x7ff8680c2730 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 msgr2=0x7ff85406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 0x7ff85406e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff86815bca0 tx=0x7ff85c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 msgr2=0x7ff8681503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff85800c460 tx=0x7ff85800c920 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 shutdown_connections 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff85406be60 0x7ff85406e320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff868155180 0x7ff86814e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8680c8f90 0x7ff868073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 --2- 172.21.15.49:0/2941425192 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff8680c2730 0x7ff8681503d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 >> 172.21.15.49:0/2941425192 conn(0x7ff8680c9650 msgr2=0x7ff8680cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:04.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 shutdown_connections 2024-05-09T08:56:04.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.699+0000 7ff86e110640 1 -- 172.21.15.49:0/2941425192 wait complete. 2024-05-09T08:56:04.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T08:56:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.825+0000 7f6457763640 1 Processor -- start 2024-05-09T08:56:04.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.826+0000 7f6457763640 1 -- start start 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6457763640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f64500c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6457763640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64500c6fd0 con 0x7f6450154b90 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6457763640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6450072600 con 0x7f64500c6b80 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6457763640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f64500728e0 con 0x7f6450154f70 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f64554ff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6450154b90 0x7f64500cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45870/0 (socket says 172.21.15.49:45870) 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f64554ff640 1 -- 172.21.15.49:0/1545428162 learned_addr learned my addr 172.21.15.49:0/1545428162 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2489725526 0 0) 0x7f64500c6fd0 con 0x7f6450154b90 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f643c003520 con 0x7f6450154b90 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6454cfe640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f64500c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2828495659 0 0) 0x7f643c003520 con 0x7f6450154b90 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 msgr2=0x7f64500c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f64500c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 >> v1:172.21.15.183:6789/0 conn(0x7f64500c6b80 legacy=0x7f6450071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64500c5880 con 0x7f6450154b90 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f644c001670 con 0x7f6450154b90 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 >> v1:172.21.15.49:6789/0 conn(0x7f6450154b90 legacy=0x7f64500cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c0039e0 0x7f643c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f643c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c004d40 0x7f643c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64500728e0 con 0x7f643c004d40 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f643c00bb00 con 0x7f643c0039e0 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f643c00bd00 con 0x7f6450154f70 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.827+0000 7f6455d00640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c004d40 0x7f643c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f64554ff640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c0039e0 0x7f643c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6454cfe640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f643c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 -- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 msgr2=0x7f643c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6450154f70 0x7f643c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 -- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c0039e0 msgr2=0x7f643c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f643c0039e0 0x7f643c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 -- 172.21.15.49:0/1545428162 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c003520 con 0x7f643c004d40 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.828+0000 7f6455d00640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c004d40 0x7f643c0051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6448001e20 tx=0x7f643c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a92fc014573b4f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448001a70 con 0x7f643c004d40 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644802ae40 con 0x7f643c004d40 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6447fff640 1 -- 172.21.15.49:0/1545428162 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448003bb0 con 0x7f643c004d40 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 -- 172.21.15.49:0/1545428162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c004d40 msgr2=0x7f643c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 --2- 172.21.15.49:0/1545428162 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f643c004d40 0x7f643c0051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6448001e20 tx=0x7f643c005830 comp rx=0 tx=0).stop 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6455d00640 1 -- 172.21.15.49:0/1545428162 reap_dead start 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 -- 172.21.15.49:0/1545428162 shutdown_connections 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 -- 172.21.15.49:0/1545428162 >> 172.21.15.49:0/1545428162 conn(0x7f6450074840 msgr2=0x7f6450074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:04.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 -- 172.21.15.49:0/1545428162 shutdown_connections 2024-05-09T08:56:04.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.829+0000 7f6457763640 1 -- 172.21.15.49:0/1545428162 wait complete. 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 Processor -- start 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 -- start start 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f643c004d40 0x7f645014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645014caa0 0x7f645014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64500c6120 con 0x7f64500c7e10 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64500c5e40 con 0x7f645014caa0 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6457763640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f645015af40 con 0x7f643c004d40 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6454cfe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6454cfe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47826/0 (socket says 172.21.15.49:47826) 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.831+0000 7f6454cfe640 1 -- 172.21.15.49:0/4269306484 learned_addr learned my addr 172.21.15.49:0/4269306484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f64554ff640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f643c004d40 0x7f645014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6455d00640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645014caa0 0x7f645014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 -- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f643c004d40 msgr2=0x7f645014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f643c004d40 0x7f645014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 -- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645014caa0 msgr2=0x7f645014cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645014caa0 0x7f645014cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 -- 172.21.15.49:0/4269306484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645014d460 con 0x7f64500c7e10 2024-05-09T08:56:04.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6454cfe640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6440011a00 tx=0x7f6440011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644000fb50 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644000fcf0 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64501f1540 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644000fea0 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.832+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64500b6110 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.833+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450159830 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.834+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6440012040 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.834+0000 7f6445ffb640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 0x7f643406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.834+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f644008b400 con 0x7f64500c7e10 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.835+0000 7f64554ff640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 0x7f643406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:04.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.835+0000 7f64554ff640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 0x7f643406e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c007fd0 tx=0x7f644c0078f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:04.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:04.839+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6450159830 con 0x7f64500c7e10 2024-05-09T08:56:05.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.061+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f645004f660 con 0x7f64500c7e10 2024-05-09T08:56:05.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.697+0000 7f6445ffb640 1 -- 172.21.15.49:0/4269306484 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f645014d460 con 0x7f64500c7e10 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 msgr2=0x7f643406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 0x7f643406e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c007fd0 tx=0x7f644c0078f0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 msgr2=0x7f645014a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6440011a00 tx=0x7f6440011ec0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 shutdown_connections 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f643406be30 0x7f643406e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645014caa0 0x7f645014cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64500c7e10 0x7f645014a550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 --2- 172.21.15.49:0/4269306484 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f643c004d40 0x7f645014a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 >> 172.21.15.49:0/4269306484 conn(0x7f6450074840 msgr2=0x7f6450159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 shutdown_connections 2024-05-09T08:56:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.700+0000 7f6457763640 1 -- 172.21.15.49:0/4269306484 wait complete. 2024-05-09T08:56:05.713 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-09T08:56:05.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.827+0000 7f17d039e640 1 Processor -- start 2024-05-09T08:56:05.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.827+0000 7f17d039e640 1 -- start start 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17d039e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f17c80cca20 0x7f17c815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17d039e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c815afa0 con 0x7f17c80cdcd0 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17d039e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c815b280 con 0x7f17c8154ea0 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17d039e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f17c815ad70 con 0x7f17c80cca20 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17cd939640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f17c80cdcd0 0x7f17c8072230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45876/0 (socket says 172.21.15.49:45876) 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17cd939640 1 -- 172.21.15.49:0/752089812 learned_addr learned my addr 172.21.15.49:0/752089812 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365595660 0 0) 0x7f17c815afa0 con 0x7f17c80cdcd0 2024-05-09T08:56:05.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17b4003520 con 0x7f17c80cdcd0 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.828+0000 7f17ce93b640 1 --2- 172.21.15.49:0/752089812 >> v2:172.21.15.183:3300/0 conn(0x7f17c80cca20 0x7f17c815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3450002441 0 0) 0x7f17b4003520 con 0x7f17c80cdcd0 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 >> v2:172.21.15.183:3300/0 conn(0x7f17c80cca20 msgr2=0x7f17c815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 --2- 172.21.15.49:0/752089812 >> v2:172.21.15.183:3300/0 conn(0x7f17c80cca20 0x7f17c815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 >> v1:172.21.15.183:6789/0 conn(0x7f17c8154ea0 legacy=0x7f17c80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c815b560 con 0x7f17c80cdcd0 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f17b8001670 con 0x7f17c80cdcd0 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 >> v1:172.21.15.49:6789/0 conn(0x7f17c80cdcd0 legacy=0x7f17c8072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b40039b0 0x7f17b4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 0x7f17b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b40051c0 0x7f17b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17b400c0c0 con 0x7f17b4004800 2024-05-09T08:56:05.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17b400c270 con 0x7f17b40039b0 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f17b400c470 con 0x7f17b40051c0 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 0x7f17b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 -- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b40051c0 msgr2=0x7f17b400b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b40051c0 0x7f17b400b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 -- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b40039b0 msgr2=0x7f17b4003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17b40039b0 0x7f17b4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17cd939640 1 -- 172.21.15.49:0/752089812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b4003520 con 0x7f17b4004800 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.829+0000 7f17ce93b640 1 -- 172.21.15.49:0/752089812 reap_dead start 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17cd939640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 0x7f17b4004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17c80d1500 tx=0x7f17b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3ddfc713689473 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8007d30 con 0x7f17b4004800 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b8005af0 con 0x7f17b4004800 2024-05-09T08:56:05.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17cd138640 1 -- 172.21.15.49:0/752089812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8007070 con 0x7f17b4004800 2024-05-09T08:56:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17d039e640 1 -- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 msgr2=0x7f17b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17d039e640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 0x7f17b4004c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f17c80d1500 tx=0x7f17b4005830 comp rx=0 tx=0).stop 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17d039e640 1 -- 172.21.15.49:0/752089812 shutdown_connections 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17d039e640 1 --2- 172.21.15.49:0/752089812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17b4004800 0x7f17b4004c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.830+0000 7f17d039e640 1 -- 172.21.15.49:0/752089812 >> 172.21.15.49:0/752089812 conn(0x7f17c8071a00 msgr2=0x7f17c8067760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.831+0000 7f17d039e640 1 -- 172.21.15.49:0/752089812 shutdown_connections 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.831+0000 7f17d039e640 1 -- 172.21.15.49:0/752089812 wait complete. 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 Processor -- start 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 -- start start 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b4004800 0x7f17c81f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c815bdc0 0x7f17c81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f17c8155280 con 0x7f17c8152f10 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17c815bb20 con 0x7f17c815bdc0 2024-05-09T08:56:05.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17d039e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f17c8155400 con 0x7f17b4004800 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.832+0000 7f17ce93b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47842/0 (socket says 172.21.15.49:47842) 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 -- 172.21.15.49:0/343186668 learned_addr learned my addr 172.21.15.49:0/343186668 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce13a640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b4004800 0x7f17c81f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17cd939640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c815bdc0 0x7f17c81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 -- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b4004800 msgr2=0x7f17c81f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b4004800 0x7f17c81f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 -- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c815bdc0 msgr2=0x7f17c81471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c815bdc0 0x7f17c81471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 -- 172.21.15.49:0/343186668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c815c5e0 con 0x7f17c8152f10 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17ce93b640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17c80cced0 tx=0x7f17c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c000ed60 con 0x7f17c8152f10 2024-05-09T08:56:05.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c0019070 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c815cdc0 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0014660 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.833+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c815d380 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.834+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c815afa0 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.835+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17c0014800 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.835+0000 7f17beffd640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 0x7f17ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.835+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17c008b300 con 0x7f17c8152f10 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.835+0000 7f17ce13a640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 0x7f17ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:05.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.836+0000 7f17ce13a640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 0x7f17ac06e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17c4003730 tx=0x7f17c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:05.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:05.840+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f17c815afa0 con 0x7f17c8152f10 2024-05-09T08:56:06.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.064+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f17c815efd0 con 0x7f17c8152f10 2024-05-09T08:56:06.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.705+0000 7f17beffd640 1 -- 172.21.15.49:0/343186668 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f17c0059c20 con 0x7f17c8152f10 2024-05-09T08:56:06.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 msgr2=0x7f17ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 0x7f17ac06e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17c4003730 tx=0x7f17c4009040 comp rx=0 tx=0).stop 2024-05-09T08:56:06.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 msgr2=0x7f17c80cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f17c80cced0 tx=0x7f17c000e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 shutdown_connections 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f17ac06be60 0x7f17ac06e320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f17c8152f10 0x7f17c80cf4f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c815bdc0 0x7f17c81471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 --2- 172.21.15.49:0/343186668 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17b4004800 0x7f17c81f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 >> 172.21.15.49:0/343186668 conn(0x7f17c8071a00 msgr2=0x7f17c81482d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 shutdown_connections 2024-05-09T08:56:06.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.708+0000 7f17d039e640 1 -- 172.21.15.49:0/343186668 wait complete. 2024-05-09T08:56:06.721 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T08:56:06.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:06.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.830+0000 7efce0dee640 1 Processor -- start 2024-05-09T08:56:06.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.830+0000 7efce0dee640 1 -- start start 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efce0dee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcdc15b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efce0dee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc15b9d0 con 0x7efcdc154f10 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efce0dee640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcdc06f900 con 0x7efcdc071bc0 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efce0dee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc072000 con 0x7efcdc15b000 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efcda59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7efcdc154f10 0x7efcdc06f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45886/0 (socket says 172.21.15.49:45886) 2024-05-09T08:56:06.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.831+0000 7efcda59c640 1 -- 172.21.15.49:0/3689552713 learned_addr learned my addr 172.21.15.49:0/3689552713 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716473676 0 0) 0x7efcdc15b9d0 con 0x7efcdc154f10 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcc8003520 con 0x7efcdc154f10 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcdc15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1929772785 0 0) 0x7efcc8003520 con 0x7efcdc154f10 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 msgr2=0x7efcdc15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcdc15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 >> v1:172.21.15.183:6789/0 conn(0x7efcdc071bc0 legacy=0x7efcdc0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc15e340 con 0x7efcdc154f10 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7efcd4001670 con 0x7efcdc154f10 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcdc15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 >> v1:172.21.15.49:6789/0 conn(0x7efcdc154f10 legacy=0x7efcdc06f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcc80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8003fd0 0x7efcc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8004d40 0x7efcc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc072000 con 0x7efcc8004d40 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efcc800bb20 con 0x7efcc8003fd0 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efcc800bd20 con 0x7efcdc15b000 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8004d40 0x7efcc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcda59c640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcc80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcd9d9b640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8003fd0 0x7efcc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 -- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 msgr2=0x7efcc80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc15b000 0x7efcc80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 -- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8003fd0 msgr2=0x7efcc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8003fd0 0x7efcc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.832+0000 7efcdad9d640 1 -- 172.21.15.49:0/3689552713 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc8003520 con 0x7efcc8004d40 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.833+0000 7efcdad9d640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8004d40 0x7efcc80051c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efccc0073e0 tx=0x7efcc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f580e117ed2ef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.833+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc009c80 con 0x7efcc8004d40 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.833+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efccc02e660 con 0x7efcc8004d40 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.833+0000 7efcd959a640 1 -- 172.21.15.49:0/3689552713 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc02ea10 con 0x7efcc8004d40 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 -- 172.21.15.49:0/3689552713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8004d40 msgr2=0x7efcc80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 --2- 172.21.15.49:0/3689552713 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcc8004d40 0x7efcc80051c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efccc0073e0 tx=0x7efcc8005830 comp rx=0 tx=0).stop 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efcdad9d640 1 -- 172.21.15.49:0/3689552713 reap_dead start 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 -- 172.21.15.49:0/3689552713 shutdown_connections 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 -- 172.21.15.49:0/3689552713 >> 172.21.15.49:0/3689552713 conn(0x7efcdc06b3e0 msgr2=0x7efcdc06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 -- 172.21.15.49:0/3689552713 shutdown_connections 2024-05-09T08:56:06.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.834+0000 7efce0dee640 1 -- 172.21.15.49:0/3689552713 wait complete. 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.835+0000 7efce0dee640 1 Processor -- start 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.835+0000 7efce0dee640 1 -- start start 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8004d40 0x7efcdc1f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc146e60 0x7efcdc1472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc15ebe0 con 0x7efcdc151230 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc15e900 con 0x7efcc8004d40 2024-05-09T08:56:06.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efce0dee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efcdc15eec0 con 0x7efcdc146e60 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47862/0 (socket says 172.21.15.49:47862) 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 -- 172.21.15.49:0/2422026823 learned_addr learned my addr 172.21.15.49:0/2422026823 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcda59c640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8004d40 0x7efcdc1f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcdad9d640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc146e60 0x7efcdc1472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 -- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc146e60 msgr2=0x7efcdc1472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc146e60 0x7efcdc1472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 -- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8004d40 msgr2=0x7efcdc1f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8004d40 0x7efcdc1f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.836+0000 7efcd9d9b640 1 -- 172.21.15.49:0/2422026823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc1f5180 con 0x7efcdc151230 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efcd9d9b640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcc400f650 tx=0x7efcc400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc40050d0 con 0x7efcdc151230 2024-05-09T08:56:06.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc4018070 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc158010 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400d730 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.837+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc1588e0 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.838+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc068560 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.838+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcc400d8d0 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.839+0000 7efcd2ffd640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 0x7efcbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.839+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efcc408ad00 con 0x7efcdc151230 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.839+0000 7efcda59c640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 0x7efcbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:06.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.840+0000 7efcda59c640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 0x7efcbc06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcd4001eb0 tx=0x7efcd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:06.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:06.843+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efcdc1f5180 con 0x7efcdc151230 2024-05-09T08:56:07.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.065+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcdc0c9000 con 0x7efcdc151230 2024-05-09T08:56:07.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.066+0000 7efcd2ffd640 1 -- 172.21.15.49:0/2422026823 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+6846 (secure 0 0 0) 0x7efcc4059620 con 0x7efcdc151230 2024-05-09T08:56:07.065 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:07.065 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:06.708764+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4285,"mds_1":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-05-09T08:56:07.065 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 9 2024-05-09T08:56:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 msgr2=0x7efcbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 0x7efcbc06e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcd4001eb0 tx=0x7efcd4009040 comp rx=0 tx=0).stop 2024-05-09T08:56:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 msgr2=0x7efcdc156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcc400f650 tx=0x7efcc400fb10 comp rx=0 tx=0).stop 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 shutdown_connections 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efcbc06be30 0x7efcbc06e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcdc146e60 0x7efcdc1472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efcdc151230 0x7efcdc156190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 --2- 172.21.15.49:0/2422026823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc8004d40 0x7efcdc1f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 >> 172.21.15.49:0/2422026823 conn(0x7efcdc06b3e0 msgr2=0x7efcdc0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 shutdown_connections 2024-05-09T08:56:07.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.068+0000 7efce0dee640 1 -- 172.21.15.49:0/2422026823 wait complete. 2024-05-09T08:56:07.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:07.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.182+0000 7f23042f5640 1 Processor -- start 2024-05-09T08:56:07.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.182+0000 7f23042f5640 1 -- start start 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f23042f5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22fc1517f0 0x7f22fc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f23042f5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f22fc0cc190 0x7f22fc0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f23042f5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22fc0ccb20 con 0x7f22fc075200 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f23042f5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc151bd0 con 0x7f22fc1517f0 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f23042f5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc15e260 con 0x7f22fc0cc190 2024-05-09T08:56:07.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f2302091640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22fc1517f0 0x7f22fc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f2302091640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22fc1517f0 0x7f22fc15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47878/0 (socket says 172.21.15.49:47878) 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.183+0000 7f2302091640 1 -- 172.21.15.49:0/3761052131 learned_addr learned my addr 172.21.15.49:0/3761052131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302892640 1 --2- 172.21.15.49:0/3761052131 >> v2:172.21.15.183:3300/0 conn(0x7f22fc0cc190 0x7f22fc0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302091640 1 -- 172.21.15.49:0/3761052131 >> v2:172.21.15.183:3300/0 conn(0x7f22fc0cc190 msgr2=0x7f22fc0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302091640 1 --2- 172.21.15.49:0/3761052131 >> v2:172.21.15.183:3300/0 conn(0x7f22fc0cc190 0x7f22fc0cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302091640 1 -- 172.21.15.49:0/3761052131 >> v1:172.21.15.183:6789/0 conn(0x7f22fc075200 legacy=0x7f22fc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:07.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302091640 1 -- 172.21.15.49:0/3761052131 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc15e520 con 0x7f22fc1517f0 2024-05-09T08:56:07.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.184+0000 7f2302091640 1 --2- 172.21.15.49:0/3761052131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc1517f0 0x7f22fc15b6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22fc158840 tx=0x7f22f402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9ae67366b419f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f230108f640 1 -- 172.21.15.49:0/3761052131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4037070 con 0x7f22fc1517f0 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f230108f640 1 -- 172.21.15.49:0/3761052131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f4032440 con 0x7f22fc1517f0 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f230108f640 1 -- 172.21.15.49:0/3761052131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4032810 con 0x7f22fc1517f0 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 -- 172.21.15.49:0/3761052131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc1517f0 msgr2=0x7f22fc15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 --2- 172.21.15.49:0/3761052131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc1517f0 0x7f22fc15b6c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22fc158840 tx=0x7f22f402a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 -- 172.21.15.49:0/3761052131 shutdown_connections 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 --2- 172.21.15.49:0/3761052131 >> v2:172.21.15.183:3300/0 conn(0x7f22fc0cc190 0x7f22fc0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 --2- 172.21.15.49:0/3761052131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc1517f0 0x7f22fc15b6c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 -- 172.21.15.49:0/3761052131 >> 172.21.15.49:0/3761052131 conn(0x7f22fc06bd00 msgr2=0x7f22fc074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 -- 172.21.15.49:0/3761052131 shutdown_connections 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.185+0000 7f23042f5640 1 -- 172.21.15.49:0/3761052131 wait complete. 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.186+0000 7f23042f5640 1 Processor -- start 2024-05-09T08:56:07.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 -- start start 2024-05-09T08:56:07.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22fc075200 0x7f22fc0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22fc1517f0 0x7f22fc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc15edc0 con 0x7f22fc0cc190 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc15eae0 con 0x7f22fc1517f0 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f23042f5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc15f0a0 con 0x7f22fc075200 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f2301890640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f2301890640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47888/0 (socket says 172.21.15.49:47888) 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.187+0000 7f2301890640 1 -- 172.21.15.49:0/4181868068 learned_addr learned my addr 172.21.15.49:0/4181868068 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2302091640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22fc075200 0x7f22fc0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2302892640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22fc1517f0 0x7f22fc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 -- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22fc075200 msgr2=0x7f22fc0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22fc075200 0x7f22fc0d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 -- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22fc1517f0 msgr2=0x7f22fc0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22fc1517f0 0x7f22fc0cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 -- 172.21.15.49:0/4181868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc1f4ed0 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f2301890640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c430 tx=0x7f22f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011790 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0017070 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011bb0 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc14fed0 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.188+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc1e4410 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.189+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc1e4610 con 0x7f22fc0cc190 2024-05-09T08:56:07.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.189+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22f001c460 con 0x7f22fc0cc190 2024-05-09T08:56:07.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.190+0000 7f22faffd640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 0x7f22e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.191 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.190+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f22f008ace0 con 0x7f22fc0cc190 2024-05-09T08:56:07.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.190+0000 7f2302091640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 0x7f22e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.191+0000 7f2302091640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 0x7f22e806e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22f4034000 tx=0x7f22f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.193 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.194+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22fc1e4610 con 0x7f22fc0cc190 2024-05-09T08:56:07.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.379+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22fc1509f0 con 0x7f22fc0cc190 2024-05-09T08:56:07.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.380+0000 7f22faffd640 1 -- 172.21.15.49:0/4181868068 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f22fc1509f0 con 0x7f22fc0cc190 2024-05-09T08:56:07.379 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:07.379 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:07.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 msgr2=0x7f22e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 0x7f22e806e320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f22f4034000 tx=0x7f22f402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:07.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 msgr2=0x7f22fc1f4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c430 tx=0x7f22f000c8f0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 shutdown_connections 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22e806be60 0x7f22e806e320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22fc1517f0 0x7f22fc0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22fc0cc190 0x7f22fc1f4990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 --2- 172.21.15.49:0/4181868068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22fc075200 0x7f22fc0d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 >> 172.21.15.49:0/4181868068 conn(0x7f22fc06bd00 msgr2=0x7f22fc14eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 shutdown_connections 2024-05-09T08:56:07.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.382+0000 7f23042f5640 1 -- 172.21.15.49:0/4181868068 wait complete. 2024-05-09T08:56:07.392 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T08:56:07.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:07.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.497+0000 7ff7283a5640 1 Processor -- start 2024-05-09T08:56:07.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.497+0000 7ff7283a5640 1 -- start start 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff7283a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff720070010 0x7ff7200c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff7283a5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff7200703f0 0x7ff72015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff7283a5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72015e360 con 0x7ff72015f030 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff7283a5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff72015bcb0 con 0x7ff7200703f0 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff7283a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff72015e660 con 0x7ff720070010 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff725940640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff7200703f0 0x7ff72015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff726141640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff720070010 0x7ff7200c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff725940640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff7200703f0 0x7ff72015eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55216/0 (socket says 172.21.15.49:55216) 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff725940640 1 -- 172.21.15.49:0/2706815761 learned_addr learned my addr 172.21.15.49:0/2706815761 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.498+0000 7ff726942640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7ff72015f030 0x7ff720154a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36330/0 (socket says 172.21.15.49:36330) 2024-05-09T08:56:07.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.499+0000 7ff725940640 1 -- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff720070010 msgr2=0x7ff7200c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.499+0000 7ff725940640 1 --2- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff720070010 0x7ff7200c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.499+0000 7ff725940640 1 -- 172.21.15.49:0/2706815761 >> v1:172.21.15.183:6789/0 conn(0x7ff72015f030 legacy=0x7ff720154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.499+0000 7ff725940640 1 -- 172.21.15.49:0/2706815761 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72006b330 con 0x7ff7200703f0 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff725940640 1 --2- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72015eaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fb71705bbed763 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff72513f640 1 -- 172.21.15.49:0/2706815761 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710037070 con 0x7ff7200703f0 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff72513f640 1 -- 172.21.15.49:0/2706815761 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710032440 con 0x7ff7200703f0 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff72513f640 1 -- 172.21.15.49:0/2706815761 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710032810 con 0x7ff7200703f0 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff7283a5640 1 -- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 msgr2=0x7ff72015eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.500+0000 7ff7283a5640 1 --2- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72015eaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71002a270 comp rx=0 tx=0).stop 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 -- 172.21.15.49:0/2706815761 shutdown_connections 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 --2- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72015eaf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 --2- 172.21.15.49:0/2706815761 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff720070010 0x7ff7200c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 -- 172.21.15.49:0/2706815761 >> 172.21.15.49:0/2706815761 conn(0x7ff720074840 msgr2=0x7ff720074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 -- 172.21.15.49:0/2706815761 shutdown_connections 2024-05-09T08:56:07.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.501+0000 7ff7283a5640 1 -- 172.21.15.49:0/2706815761 wait complete. 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 Processor -- start 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 -- start start 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72014a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 0x7ff72014a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff72006b8f0 con 0x7ff720070010 2024-05-09T08:56:07.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff72015c1c0 con 0x7ff7200703f0 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.502+0000 7ff7283a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff72006ba70 con 0x7ff72015f030 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47902/0 (socket says 172.21.15.49:47902) 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 -- 172.21.15.49:0/4131247022 learned_addr learned my addr 172.21.15.49:0/4131247022 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726942640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 0x7ff72014a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 -- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 msgr2=0x7ff72014a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 0x7ff72014a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 -- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 msgr2=0x7ff72014a290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72014a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 -- 172.21.15.49:0/4131247022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72014af40 con 0x7ff720070010 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726942640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 0x7ff72014a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff726141640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9e0 tx=0x7ff71c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c017070 con 0x7ff720070010 2024-05-09T08:56:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c00c9d0 con 0x7ff720070010 2024-05-09T08:56:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201f4cc0 con 0x7ff720070010 2024-05-09T08:56:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00cdc0 con 0x7ff720070010 2024-05-09T08:56:07.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.503+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff720151810 con 0x7ff720070010 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.504+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72006c7a0 con 0x7ff720070010 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.505+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff71c01a070 con 0x7ff720070010 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.505+0000 7ff716ffd640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.505+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff71c08b230 con 0x7ff720070010 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.506+0000 7ff725940640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.507+0000 7ff725940640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 0x7ff70406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.510+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff72006c7a0 con 0x7ff720070010 2024-05-09T08:56:07.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.689+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff720068560 con 0x7ff70406be30 2024-05-09T08:56:07.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.696+0000 7ff716ffd640 1 -- 172.21.15.49:0/4131247022 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251851 (secure 0 0 0) 0x7ff720068560 con 0x7ff70406be30 2024-05-09T08:56:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:07.699 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:07.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 msgr2=0x7ff70406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 0x7ff70406e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff710034000 tx=0x7ff71002e000 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 msgr2=0x7ff7200cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9e0 tx=0x7ff71c00eea0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 shutdown_connections 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff70406be30 0x7ff70406e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff72015f030 0x7ff72014a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7200703f0 0x7ff72014a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 --2- 172.21.15.49:0/4131247022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff720070010 0x7ff7200cdb80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 >> 172.21.15.49:0/4131247022 conn(0x7ff720074840 msgr2=0x7ff7200cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 shutdown_connections 2024-05-09T08:56:07.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.699+0000 7ff7283a5640 1 -- 172.21.15.49:0/4131247022 wait complete. 2024-05-09T08:56:07.710 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-09T08:56:06.572420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235584,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739136,"statfs":{"total":773094113280,"available":772852875264,"internally_reserved":0,"allocated":2826240,"data_stored":1748208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.898471"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651794+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:03.651794+0000","last_peered":"2024-05-09T08:56:03.651794+0000","last_clean":"2024-05-09T08:56:03.651794+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:03.651794+0000","last_undegraded":"2024-05-09T08:56:03.651794+0000","last_fullsized":"2024-05-09T08:56:03.651794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005273+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:03.005273+0000","last_peered":"2024-05-09T08:56:03.005273+0000","last_clean":"2024-05-09T08:56:03.005273+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:03.005273+0000","last_undegraded":"2024-05-09T08:56:03.005273+0000","last_fullsized":"2024-05-09T08:56:03.005273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651547+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:03.651547+0000","last_peered":"2024-05-09T08:56:03.651547+0000","last_clean":"2024-05-09T08:56:03.651547+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:03.651547+0000","last_undegraded":"2024-05-09T08:56:03.651547+0000","last_fullsized":"2024-05-09T08:56:03.651547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077137+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:03.077137+0000","last_peered":"2024-05-09T08:56:03.077137+0000","last_clean":"2024-05-09T08:56:03.077137+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:03.077137+0000","last_undegraded":"2024-05-09T08:56:03.077137+0000","last_fullsized":"2024-05-09T08:56:03.077137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005453+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:03.005453+0000","last_peered":"2024-05-09T08:56:03.005453+0000","last_clean":"2024-05-09T08:56:03.005453+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:03.005453+0000","last_undegraded":"2024-05-09T08:56:03.005453+0000","last_fullsized":"2024-05-09T08:56:03.005453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077401+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:03.077401+0000","last_peered":"2024-05-09T08:56:03.077401+0000","last_clean":"2024-05-09T08:56:03.077401+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:03.077401+0000","last_undegraded":"2024-05-09T08:56:03.077401+0000","last_fullsized":"2024-05-09T08:56:03.077401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693446+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:02.693446+0000","last_peered":"2024-05-09T08:56:02.693446+0000","last_clean":"2024-05-09T08:56:02.693446+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:02.693446+0000","last_undegraded":"2024-05-09T08:56:02.693446+0000","last_fullsized":"2024-05-09T08:56:02.693446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005542+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:03.005542+0000","last_peered":"2024-05-09T08:56:03.005542+0000","last_clean":"2024-05-09T08:56:03.005542+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:03.005542+0000","last_undegraded":"2024-05-09T08:56:03.005542+0000","last_fullsized":"2024-05-09T08:56:03.005542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043247+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:03.043247+0000","last_peered":"2024-05-09T08:56:03.043247+0000","last_clean":"2024-05-09T08:56:03.043247+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:03.043247+0000","last_undegraded":"2024-05-09T08:56:03.043247+0000","last_fullsized":"2024-05-09T08:56:03.043247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077321+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:03.077321+0000","last_peered":"2024-05-09T08:56:03.077321+0000","last_clean":"2024-05-09T08:56:03.077321+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:03.077321+0000","last_undegraded":"2024-05-09T08:56:03.077321+0000","last_fullsized":"2024-05-09T08:56:03.077321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005220+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:03.005220+0000","last_peered":"2024-05-09T08:56:03.005220+0000","last_clean":"2024-05-09T08:56:03.005220+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:03.005220+0000","last_undegraded":"2024-05-09T08:56:03.005220+0000","last_fullsized":"2024-05-09T08:56:03.005220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688155+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:02.688155+0000","last_peered":"2024-05-09T08:56:02.688155+0000","last_clean":"2024-05-09T08:56:02.688155+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:02.688155+0000","last_undegraded":"2024-05-09T08:56:02.688155+0000","last_fullsized":"2024-05-09T08:56:02.688155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695572+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:02.695572+0000","last_peered":"2024-05-09T08:56:02.695572+0000","last_clean":"2024-05-09T08:56:02.695572+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:02.695572+0000","last_undegraded":"2024-05-09T08:56:02.695572+0000","last_fullsized":"2024-05-09T08:56:02.695572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688378+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:02.688378+0000","last_peered":"2024-05-09T08:56:02.688378+0000","last_clean":"2024-05-09T08:56:02.688378+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:02.688378+0000","last_undegraded":"2024-05-09T08:56:02.688378+0000","last_fullsized":"2024-05-09T08:56:02.688378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693482+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:02.693482+0000","last_peered":"2024-05-09T08:56:02.693482+0000","last_clean":"2024-05-09T08:56:02.693482+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:02.693482+0000","last_undegraded":"2024-05-09T08:56:02.693482+0000","last_fullsized":"2024-05-09T08:56:02.693482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688423+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:02.688423+0000","last_peered":"2024-05-09T08:56:02.688423+0000","last_clean":"2024-05-09T08:56:02.688423+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:02.688423+0000","last_undegraded":"2024-05-09T08:56:02.688423+0000","last_fullsized":"2024-05-09T08:56:02.688423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077019+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:03.077019+0000","last_peered":"2024-05-09T08:56:03.077019+0000","last_clean":"2024-05-09T08:56:03.077019+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:03.077019+0000","last_undegraded":"2024-05-09T08:56:03.077019+0000","last_fullsized":"2024-05-09T08:56:03.077019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651254+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:03.651254+0000","last_peered":"2024-05-09T08:56:03.651254+0000","last_clean":"2024-05-09T08:56:03.651254+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:03.651254+0000","last_undegraded":"2024-05-09T08:56:03.651254+0000","last_fullsized":"2024-05-09T08:56:03.651254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688243+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:02.688243+0000","last_peered":"2024-05-09T08:56:02.688243+0000","last_clean":"2024-05-09T08:56:02.688243+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:02.688243+0000","last_undegraded":"2024-05-09T08:56:02.688243+0000","last_fullsized":"2024-05-09T08:56:02.688243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077432+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:03.077432+0000","last_peered":"2024-05-09T08:56:03.077432+0000","last_clean":"2024-05-09T08:56:03.077432+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:03.077432+0000","last_undegraded":"2024-05-09T08:56:03.077432+0000","last_fullsized":"2024-05-09T08:56:03.077432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693243+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693243+0000","last_peered":"2024-05-09T08:56:02.693243+0000","last_clean":"2024-05-09T08:56:02.693243+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693243+0000","last_undegraded":"2024-05-09T08:56:02.693243+0000","last_fullsized":"2024-05-09T08:56:02.693243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651913+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:03.651913+0000","last_peered":"2024-05-09T08:56:03.651913+0000","last_clean":"2024-05-09T08:56:03.651913+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:03.651913+0000","last_undegraded":"2024-05-09T08:56:03.651913+0000","last_fullsized":"2024-05-09T08:56:03.651913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693454+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:02.693454+0000","last_peered":"2024-05-09T08:56:02.693454+0000","last_clean":"2024-05-09T08:56:02.693454+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:02.693454+0000","last_undegraded":"2024-05-09T08:56:02.693454+0000","last_fullsized":"2024-05-09T08:56:02.693454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005571+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:03.005571+0000","last_peered":"2024-05-09T08:56:03.005571+0000","last_clean":"2024-05-09T08:56:03.005571+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:03.005571+0000","last_undegraded":"2024-05-09T08:56:03.005571+0000","last_fullsized":"2024-05-09T08:56:03.005571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651234+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:03.651234+0000","last_peered":"2024-05-09T08:56:03.651234+0000","last_clean":"2024-05-09T08:56:03.651234+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:03.651234+0000","last_undegraded":"2024-05-09T08:56:03.651234+0000","last_fullsized":"2024-05-09T08:56:03.651234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693333+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:02.693333+0000","last_peered":"2024-05-09T08:56:02.693333+0000","last_clean":"2024-05-09T08:56:02.693333+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:02.693333+0000","last_undegraded":"2024-05-09T08:56:02.693333+0000","last_fullsized":"2024-05-09T08:56:02.693333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693327+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:02.693327+0000","last_peered":"2024-05-09T08:56:02.693327+0000","last_clean":"2024-05-09T08:56:02.693327+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:02.693327+0000","last_undegraded":"2024-05-09T08:56:02.693327+0000","last_fullsized":"2024-05-09T08:56:02.693327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651835+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:03.651835+0000","last_peered":"2024-05-09T08:56:03.651835+0000","last_clean":"2024-05-09T08:56:03.651835+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:03.651835+0000","last_undegraded":"2024-05-09T08:56:03.651835+0000","last_fullsized":"2024-05-09T08:56:03.651835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651519+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:03.651519+0000","last_peered":"2024-05-09T08:56:03.651519+0000","last_clean":"2024-05-09T08:56:03.651519+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:03.651519+0000","last_undegraded":"2024-05-09T08:56:03.651519+0000","last_fullsized":"2024-05-09T08:56:03.651519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651774+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:03.651774+0000","last_peered":"2024-05-09T08:56:03.651774+0000","last_clean":"2024-05-09T08:56:03.651774+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:03.651774+0000","last_undegraded":"2024-05-09T08:56:03.651774+0000","last_fullsized":"2024-05-09T08:56:03.651774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652004+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:03.652004+0000","last_peered":"2024-05-09T08:56:03.652004+0000","last_clean":"2024-05-09T08:56:03.652004+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:03.652004+0000","last_undegraded":"2024-05-09T08:56:03.652004+0000","last_fullsized":"2024-05-09T08:56:03.652004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043120+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:03.043120+0000","last_peered":"2024-05-09T08:56:03.043120+0000","last_clean":"2024-05-09T08:56:03.043120+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:03.043120+0000","last_undegraded":"2024-05-09T08:56:03.043120+0000","last_fullsized":"2024-05-09T08:56:03.043120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:07.714 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:07.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.854+0000 7feda1ed6640 1 Processor -- start 2024-05-09T08:56:07.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 -- start start 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c075200 0x7fed9c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fed9c0cc190 0x7fed9c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed9c0ccb60 con 0x7fed9c1517f0 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fed9c151bd0 con 0x7fed9c0cc190 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7feda1ed6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c15e260 con 0x7fed9c075200 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7fed9b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fed9c1517f0 0x7fed9c0ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45900/0 (socket says 172.21.15.49:45900) 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.855+0000 7fed9b7fe640 1 -- 172.21.15.49:0/486297945 learned_addr learned my addr 172.21.15.49:0/486297945 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:07.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 --2- 172.21.15.49:0/486297945 >> v2:172.21.15.49:3300/0 conn(0x7fed9c0cc190 0x7fed9c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9a7fc640 1 -- 172.21.15.49:0/486297945 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869027027 0 0) 0x7fed9c0ccb60 con 0x7fed9c1517f0 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9a7fc640 1 -- 172.21.15.49:0/486297945 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed88003520 con 0x7fed9c1517f0 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 -- 172.21.15.49:0/486297945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c075200 msgr2=0x7fed9c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 --2- 172.21.15.49:0/486297945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c075200 0x7fed9c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 -- 172.21.15.49:0/486297945 >> v1:172.21.15.49:6789/0 conn(0x7fed9c1517f0 legacy=0x7fed9c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 -- 172.21.15.49:0/486297945 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c15e520 con 0x7fed9c0cc190 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.856+0000 7fed9bfff640 1 --2- 172.21.15.49:0/486297945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cc190 0x7fed9c0cc620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed940097d0 tx=0x7fed9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53089ba4682386d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7fed9a7fc640 1 -- 172.21.15.49:0/486297945 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94037070 con 0x7fed9c0cc190 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7fed9a7fc640 1 -- 172.21.15.49:0/486297945 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94032440 con 0x7fed9c0cc190 2024-05-09T08:56:07.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7fed9a7fc640 1 -- 172.21.15.49:0/486297945 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94032810 con 0x7fed9c0cc190 2024-05-09T08:56:07.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 -- 172.21.15.49:0/486297945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cc190 msgr2=0x7fed9c0cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 --2- 172.21.15.49:0/486297945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cc190 0x7fed9c0cc620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed940097d0 tx=0x7fed9402a250 comp rx=0 tx=0).stop 2024-05-09T08:56:07.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 -- 172.21.15.49:0/486297945 shutdown_connections 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 --2- 172.21.15.49:0/486297945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c0cc190 0x7fed9c0cc620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 --2- 172.21.15.49:0/486297945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c075200 0x7fed9c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 -- 172.21.15.49:0/486297945 >> 172.21.15.49:0/486297945 conn(0x7fed9c06bd00 msgr2=0x7fed9c074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 -- 172.21.15.49:0/486297945 shutdown_connections 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.857+0000 7feda1ed6640 1 -- 172.21.15.49:0/486297945 wait complete. 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.858+0000 7feda1ed6640 1 Processor -- start 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 -- start start 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c0cc190 0x7fed9c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 0x7fed9c0d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c15edc0 con 0x7fed9c075200 2024-05-09T08:56:07.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c15eae0 con 0x7fed9c0cc190 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7feda1ed6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c15f0a0 con 0x7fed9c1517f0 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7fed9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47910/0 (socket says 172.21.15.49:47910) 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.859+0000 7fed9b7fe640 1 -- 172.21.15.49:0/3562465413 learned_addr learned my addr 172.21.15.49:0/3562465413 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9bfff640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 0x7fed9c0d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9affd640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c0cc190 0x7fed9c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 -- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 msgr2=0x7fed9c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 0x7fed9c0d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 -- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c0cc190 msgr2=0x7fed9c0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c0cc190 0x7fed9c0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 -- 172.21.15.49:0/3562465413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c0ca4e0 con 0x7fed9c075200 2024-05-09T08:56:07.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9bfff640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 0x7fed9c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed9b7fe640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00cad0 tx=0x7fed8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c017070 con 0x7fed9c075200 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8c00ac30 con 0x7fed9c075200 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c0ca7c0 con 0x7fed9c075200 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8c012400 con 0x7fed9c075200 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.860+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c15dde0 con 0x7fed9c075200 2024-05-09T08:56:07.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.861+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c14ff40 con 0x7fed9c075200 2024-05-09T08:56:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.862+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed8c01a070 con 0x7fed9c075200 2024-05-09T08:56:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.862+0000 7fed93fff640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 0x7fed7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.862+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed8c08ac50 con 0x7fed9c075200 2024-05-09T08:56:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.863+0000 7fed9affd640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 0x7fed7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:07.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.863+0000 7fed9affd640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 0x7fed7c06e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0b3ff0 tx=0x7fed84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:07.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:07.866+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed9c14ff40 con 0x7fed9c075200 2024-05-09T08:56:08.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.044+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed9c068560 con 0x7fed7c06be30 2024-05-09T08:56:08.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.052+0000 7fed93fff640 1 -- 172.21.15.49:0/3562465413 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251851 (secure 0 0 0) 0x7fed9c068560 con 0x7fed7c06be30 2024-05-09T08:56:08.053 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:08.054 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:08.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 msgr2=0x7fed7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 0x7fed7c06e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0b3ff0 tx=0x7fed84008040 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 msgr2=0x7fed9c15be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00cad0 tx=0x7fed8c00cf90 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 shutdown_connections 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fed7c06be30 0x7fed7c06e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed9c1517f0 0x7fed9c0d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed9c0cc190 0x7fed9c0d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 --2- 172.21.15.49:0/3562465413 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fed9c075200 0x7fed9c15be30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 >> 172.21.15.49:0/3562465413 conn(0x7fed9c06bd00 msgr2=0x7fed9c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.054+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 shutdown_connections 2024-05-09T08:56:08.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.055+0000 7feda1ed6640 1 -- 172.21.15.49:0/3562465413 wait complete. 2024-05-09T08:56:08.064 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-09T08:56:06.572420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235584,"kb_used_data":2760,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739136,"statfs":{"total":773094113280,"available":772852875264,"internally_reserved":0,"allocated":2826240,"data_stored":1748208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.898471"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651794+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:03.651794+0000","last_peered":"2024-05-09T08:56:03.651794+0000","last_clean":"2024-05-09T08:56:03.651794+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:03.651794+0000","last_undegraded":"2024-05-09T08:56:03.651794+0000","last_fullsized":"2024-05-09T08:56:03.651794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005273+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:03.005273+0000","last_peered":"2024-05-09T08:56:03.005273+0000","last_clean":"2024-05-09T08:56:03.005273+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:03.005273+0000","last_undegraded":"2024-05-09T08:56:03.005273+0000","last_fullsized":"2024-05-09T08:56:03.005273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651547+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:03.651547+0000","last_peered":"2024-05-09T08:56:03.651547+0000","last_clean":"2024-05-09T08:56:03.651547+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:03.651547+0000","last_undegraded":"2024-05-09T08:56:03.651547+0000","last_fullsized":"2024-05-09T08:56:03.651547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077137+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:03.077137+0000","last_peered":"2024-05-09T08:56:03.077137+0000","last_clean":"2024-05-09T08:56:03.077137+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:03.077137+0000","last_undegraded":"2024-05-09T08:56:03.077137+0000","last_fullsized":"2024-05-09T08:56:03.077137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005453+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:03.005453+0000","last_peered":"2024-05-09T08:56:03.005453+0000","last_clean":"2024-05-09T08:56:03.005453+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:03.005453+0000","last_undegraded":"2024-05-09T08:56:03.005453+0000","last_fullsized":"2024-05-09T08:56:03.005453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077401+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:03.077401+0000","last_peered":"2024-05-09T08:56:03.077401+0000","last_clean":"2024-05-09T08:56:03.077401+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:03.077401+0000","last_undegraded":"2024-05-09T08:56:03.077401+0000","last_fullsized":"2024-05-09T08:56:03.077401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693446+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:02.693446+0000","last_peered":"2024-05-09T08:56:02.693446+0000","last_clean":"2024-05-09T08:56:02.693446+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:02.693446+0000","last_undegraded":"2024-05-09T08:56:02.693446+0000","last_fullsized":"2024-05-09T08:56:02.693446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005542+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:03.005542+0000","last_peered":"2024-05-09T08:56:03.005542+0000","last_clean":"2024-05-09T08:56:03.005542+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:03.005542+0000","last_undegraded":"2024-05-09T08:56:03.005542+0000","last_fullsized":"2024-05-09T08:56:03.005542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043247+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:03.043247+0000","last_peered":"2024-05-09T08:56:03.043247+0000","last_clean":"2024-05-09T08:56:03.043247+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:03.043247+0000","last_undegraded":"2024-05-09T08:56:03.043247+0000","last_fullsized":"2024-05-09T08:56:03.043247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077321+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:03.077321+0000","last_peered":"2024-05-09T08:56:03.077321+0000","last_clean":"2024-05-09T08:56:03.077321+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:03.077321+0000","last_undegraded":"2024-05-09T08:56:03.077321+0000","last_fullsized":"2024-05-09T08:56:03.077321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005220+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:03.005220+0000","last_peered":"2024-05-09T08:56:03.005220+0000","last_clean":"2024-05-09T08:56:03.005220+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:03.005220+0000","last_undegraded":"2024-05-09T08:56:03.005220+0000","last_fullsized":"2024-05-09T08:56:03.005220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688155+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:02.688155+0000","last_peered":"2024-05-09T08:56:02.688155+0000","last_clean":"2024-05-09T08:56:02.688155+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:02.688155+0000","last_undegraded":"2024-05-09T08:56:02.688155+0000","last_fullsized":"2024-05-09T08:56:02.688155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695572+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:02.695572+0000","last_peered":"2024-05-09T08:56:02.695572+0000","last_clean":"2024-05-09T08:56:02.695572+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:02.695572+0000","last_undegraded":"2024-05-09T08:56:02.695572+0000","last_fullsized":"2024-05-09T08:56:02.695572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688378+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:02.688378+0000","last_peered":"2024-05-09T08:56:02.688378+0000","last_clean":"2024-05-09T08:56:02.688378+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:02.688378+0000","last_undegraded":"2024-05-09T08:56:02.688378+0000","last_fullsized":"2024-05-09T08:56:02.688378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693482+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:02.693482+0000","last_peered":"2024-05-09T08:56:02.693482+0000","last_clean":"2024-05-09T08:56:02.693482+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:02.693482+0000","last_undegraded":"2024-05-09T08:56:02.693482+0000","last_fullsized":"2024-05-09T08:56:02.693482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688423+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:02.688423+0000","last_peered":"2024-05-09T08:56:02.688423+0000","last_clean":"2024-05-09T08:56:02.688423+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:02.688423+0000","last_undegraded":"2024-05-09T08:56:02.688423+0000","last_fullsized":"2024-05-09T08:56:02.688423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077019+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:03.077019+0000","last_peered":"2024-05-09T08:56:03.077019+0000","last_clean":"2024-05-09T08:56:03.077019+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:03.077019+0000","last_undegraded":"2024-05-09T08:56:03.077019+0000","last_fullsized":"2024-05-09T08:56:03.077019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651254+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:03.651254+0000","last_peered":"2024-05-09T08:56:03.651254+0000","last_clean":"2024-05-09T08:56:03.651254+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:03.651254+0000","last_undegraded":"2024-05-09T08:56:03.651254+0000","last_fullsized":"2024-05-09T08:56:03.651254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688243+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:02.688243+0000","last_peered":"2024-05-09T08:56:02.688243+0000","last_clean":"2024-05-09T08:56:02.688243+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:02.688243+0000","last_undegraded":"2024-05-09T08:56:02.688243+0000","last_fullsized":"2024-05-09T08:56:02.688243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077432+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:03.077432+0000","last_peered":"2024-05-09T08:56:03.077432+0000","last_clean":"2024-05-09T08:56:03.077432+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:03.077432+0000","last_undegraded":"2024-05-09T08:56:03.077432+0000","last_fullsized":"2024-05-09T08:56:03.077432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693243+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693243+0000","last_peered":"2024-05-09T08:56:02.693243+0000","last_clean":"2024-05-09T08:56:02.693243+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693243+0000","last_undegraded":"2024-05-09T08:56:02.693243+0000","last_fullsized":"2024-05-09T08:56:02.693243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651913+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:03.651913+0000","last_peered":"2024-05-09T08:56:03.651913+0000","last_clean":"2024-05-09T08:56:03.651913+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:03.651913+0000","last_undegraded":"2024-05-09T08:56:03.651913+0000","last_fullsized":"2024-05-09T08:56:03.651913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693454+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:02.693454+0000","last_peered":"2024-05-09T08:56:02.693454+0000","last_clean":"2024-05-09T08:56:02.693454+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:02.693454+0000","last_undegraded":"2024-05-09T08:56:02.693454+0000","last_fullsized":"2024-05-09T08:56:02.693454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005571+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:03.005571+0000","last_peered":"2024-05-09T08:56:03.005571+0000","last_clean":"2024-05-09T08:56:03.005571+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:03.005571+0000","last_undegraded":"2024-05-09T08:56:03.005571+0000","last_fullsized":"2024-05-09T08:56:03.005571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651234+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:03.651234+0000","last_peered":"2024-05-09T08:56:03.651234+0000","last_clean":"2024-05-09T08:56:03.651234+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:03.651234+0000","last_undegraded":"2024-05-09T08:56:03.651234+0000","last_fullsized":"2024-05-09T08:56:03.651234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693333+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:02.693333+0000","last_peered":"2024-05-09T08:56:02.693333+0000","last_clean":"2024-05-09T08:56:02.693333+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:02.693333+0000","last_undegraded":"2024-05-09T08:56:02.693333+0000","last_fullsized":"2024-05-09T08:56:02.693333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693327+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:02.693327+0000","last_peered":"2024-05-09T08:56:02.693327+0000","last_clean":"2024-05-09T08:56:02.693327+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:02.693327+0000","last_undegraded":"2024-05-09T08:56:02.693327+0000","last_fullsized":"2024-05-09T08:56:02.693327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651835+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:03.651835+0000","last_peered":"2024-05-09T08:56:03.651835+0000","last_clean":"2024-05-09T08:56:03.651835+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:03.651835+0000","last_undegraded":"2024-05-09T08:56:03.651835+0000","last_fullsized":"2024-05-09T08:56:03.651835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651519+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:03.651519+0000","last_peered":"2024-05-09T08:56:03.651519+0000","last_clean":"2024-05-09T08:56:03.651519+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:03.651519+0000","last_undegraded":"2024-05-09T08:56:03.651519+0000","last_fullsized":"2024-05-09T08:56:03.651519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651774+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:03.651774+0000","last_peered":"2024-05-09T08:56:03.651774+0000","last_clean":"2024-05-09T08:56:03.651774+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:03.651774+0000","last_undegraded":"2024-05-09T08:56:03.651774+0000","last_fullsized":"2024-05-09T08:56:03.651774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652004+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:03.652004+0000","last_peered":"2024-05-09T08:56:03.652004+0000","last_clean":"2024-05-09T08:56:03.652004+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:03.652004+0000","last_undegraded":"2024-05-09T08:56:03.652004+0000","last_fullsized":"2024-05-09T08:56:03.652004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043120+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:03.043120+0000","last_peered":"2024-05-09T08:56:03.043120+0000","last_clean":"2024-05-09T08:56:03.043120+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:03.043120+0000","last_undegraded":"2024-05-09T08:56:03.043120+0000","last_fullsized":"2024-05-09T08:56:03.043120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":798720,"data_stored":661302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:08.068 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T08:56:08.068 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T08:56:08.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T08:56:08.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T08:56:08.068 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T08:56:08.208 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.209+0000 7f7c2742c640 1 Processor -- start 2024-05-09T08:56:08.209 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.209+0000 7f7c2742c640 1 -- start start 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 0x7f7c2015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c20075200 0x7f7c200ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c200cd520 0x7f7c200cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7c20151bd0 con 0x7f7c20075200 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2015e260 con 0x7f7c201517f0 2024-05-09T08:56:08.210 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c2742c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2015e520 con 0x7f7c200cd520 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c249c7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c20075200 0x7f7c200ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c249c7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c20075200 0x7f7c200ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47920/0 (socket says 172.21.15.49:47920) 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.210+0000 7f7c249c7640 1 -- 172.21.15.49:0/3605936684 learned_addr learned my addr 172.21.15.49:0/3605936684 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c259c9640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c200cd520 0x7f7c200cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c251c8640 1 --2- 172.21.15.49:0/3605936684 >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 0x7f7c2015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 -- 172.21.15.49:0/3605936684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c200cd520 msgr2=0x7f7c200cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c200cd520 0x7f7c200cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 -- 172.21.15.49:0/3605936684 >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 msgr2=0x7f7c2015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3605936684 >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 0x7f7c2015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 -- 172.21.15.49:0/3605936684 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2015e800 con 0x7f7c20075200 2024-05-09T08:56:08.211 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c251c8640 1 --2- 172.21.15.49:0/3605936684 >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 0x7f7c2015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:08.212 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.211+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c20075200 0x7f7c200ccfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c200c78a0 tx=0x7f7c1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c1c812e168a30e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c1ffff640 1 -- 172.21.15.49:0/3605936684 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14037070 con 0x7f7c20075200 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c1ffff640 1 -- 172.21.15.49:0/3605936684 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c14032440 con 0x7f7c20075200 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c1ffff640 1 -- 172.21.15.49:0/3605936684 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14032810 con 0x7f7c20075200 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 -- 172.21.15.49:0/3605936684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c20075200 msgr2=0x7f7c200ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c20075200 0x7f7c200ccfe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c200c78a0 tx=0x7f7c1402a270 comp rx=0 tx=0).stop 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 -- 172.21.15.49:0/3605936684 shutdown_connections 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c200cd520 0x7f7c200cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3605936684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c20075200 0x7f7c200ccfe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3605936684 >> v2:172.21.15.183:3300/0 conn(0x7f7c201517f0 0x7f7c2015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 -- 172.21.15.49:0/3605936684 >> 172.21.15.49:0/3605936684 conn(0x7f7c2006bd00 msgr2=0x7f7c20074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 -- 172.21.15.49:0/3605936684 shutdown_connections 2024-05-09T08:56:08.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.212+0000 7f7c2742c640 1 -- 172.21.15.49:0/3605936684 wait complete. 2024-05-09T08:56:08.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.213+0000 7f7c2742c640 1 Processor -- start 2024-05-09T08:56:08.214 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.213+0000 7f7c2742c640 1 -- start start 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c20075200 0x7f7c201e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c201517f0 0x7f7c2014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2015f0a0 con 0x7f7c200cd520 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2015edc0 con 0x7f7c201517f0 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c2742c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c200cbc70 con 0x7f7c20075200 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c249c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c249c7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47934/0 (socket says 172.21.15.49:47934) 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.214+0000 7f7c249c7640 1 -- 172.21.15.49:0/3063105532 learned_addr learned my addr 172.21.15.49:0/3063105532 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 -- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c20075200 msgr2=0x7f7c201e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c20075200 0x7f7c201e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 -- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c201517f0 msgr2=0x7f7c2014b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c201517f0 0x7f7c2014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 -- 172.21.15.49:0/3063105532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201e4640 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c249c7640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c1402a2b0 tx=0x7f7c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14037040 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c14042070 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1403b870 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c200ce4e0 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.215+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c200ceaa0 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.216+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20068560 con 0x7f7c200cd520 2024-05-09T08:56:08.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.217+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c1403ba10 con 0x7f7c200cd520 2024-05-09T08:56:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.217+0000 7f7c1dffb640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.217+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c140ab550 con 0x7f7c200cd520 2024-05-09T08:56:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.218+0000 7f7c251c8640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.218+0000 7f7c251c8640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.222+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c14079f90 con 0x7f7c200cd520 2024-05-09T08:56:08.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.455+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7c200caa00 con 0x7f7c200cd520 2024-05-09T08:56:08.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.458+0000 7f7c1dffb640 1 -- 172.21.15.49:0/3063105532 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172944 (secure 0 0 0) 0x7f7c14079890 con 0x7f7c200cd520 2024-05-09T08:56:08.459 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 msgr2=0x7f7c0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006fd0 tx=0x7f7c18008040 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 msgr2=0x7f7c2014af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c1402a2b0 tx=0x7f7c140046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 shutdown_connections 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c0c06be30 0x7f7c0c06e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c201517f0 0x7f7c2014b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c200cd520 0x7f7c2014af10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 --2- 172.21.15.49:0/3063105532 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c20075200 0x7f7c201e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 >> 172.21.15.49:0/3063105532 conn(0x7f7c2006bd00 msgr2=0x7f7c2014fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 shutdown_connections 2024-05-09T08:56:08.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.461+0000 7f7c2742c640 1 -- 172.21.15.49:0/3063105532 wait complete. 2024-05-09T08:56:08.471 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3271117140},{"type":"v1","addr":"172.21.15.183:6833","nonce":3271117140}]},"active_addr":"172.21.15.183:6833/3271117140","active_change":"2024-05-09T08:55:48.561778+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2362494867}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":724028894}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":151072907}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2909885222}]}]} 2024-05-09T08:56:08.474 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T08:56:08.474 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T08:56:08.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.579+0000 7f9d7ef09640 1 Processor -- start 2024-05-09T08:56:08.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.579+0000 7f9d7ef09640 1 -- start start 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7ef09640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9d78151e00 0x7f9d780c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7ef09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 0x7f9d781622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7ef09640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d780c8350 con 0x7f9d780c9c00 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7ef09640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f9d781521e0 con 0x7f9d78151e00 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7ef09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9d78158490 con 0x7f9d780c8780 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d77fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9d780c9c00 0x7f9d780c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45908/0 (socket says 172.21.15.49:45908) 2024-05-09T08:56:08.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d77fff640 1 -- 172.21.15.49:0/3118589751 learned_addr learned my addr 172.21.15.49:0/3118589751 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3118589751 >> v2:172.21.15.183:3300/0 conn(0x7f9d78151e00 0x7f9d780c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 0x7f9d781622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780013981 0 0) 0x7f9d64001110 con 0x7f9d780c9c00 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d68003520 con 0x7f9d780c9c00 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3948274413 0 0) 0x7f9d68003520 con 0x7f9d780c9c00 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 msgr2=0x7f9d781622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 0x7f9d781622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 >> v2:172.21.15.183:3300/0 conn(0x7f9d78151e00 msgr2=0x7f9d780c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 --2- 172.21.15.49:0/3118589751 >> v2:172.21.15.183:3300/0 conn(0x7f9d78151e00 0x7f9d780c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d78158750 con 0x7f9d780c9c00 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.580+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f9d64001840 con 0x7f9d780c9c00 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 >> v1:172.21.15.49:6789/0 conn(0x7f9d780c9c00 legacy=0x7f9d780c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d680039b0 0x7f9d68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 0x7f9d68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d68004d40 0x7f9d680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d781521e0 con 0x7f9d68004d40 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78158490 con 0x7f9d680039b0 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6800b9c0 con 0x7f9d780c8780 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d68004d40 0x7f9d680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d680039b0 0x7f9d68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 -- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 msgr2=0x7f9d68004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d780c8780 0x7f9d68004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 -- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d680039b0 msgr2=0x7f9d68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d680039b0 0x7f9d68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7d4a6640 1 -- 172.21.15.49:0/3118589751 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d68003520 con 0x7f9d68004d40 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.581+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d680039b0 0x7f9d68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d68004d40 0x7f9d680051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009fc0 tx=0x7f9d68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eff995641766ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70034020 con 0x7f9d68004d40 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d700305f0 con 0x7f9d68004d40 2024-05-09T08:56:08.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d777fe640 1 -- 172.21.15.49:0/3118589751 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70009c10 con 0x7f9d68004d40 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3118589751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d68004d40 msgr2=0x7f9d680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3118589751 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d68004d40 0x7f9d680051c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009fc0 tx=0x7f9d68005830 comp rx=0 tx=0).stop 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7d4a6640 1 -- 172.21.15.49:0/3118589751 reap_dead start 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3118589751 shutdown_connections 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3118589751 >> 172.21.15.49:0/3118589751 conn(0x7f9d78074840 msgr2=0x7f9d78074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3118589751 shutdown_connections 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.582+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3118589751 wait complete. 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 Processor -- start 2024-05-09T08:56:08.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 -- start start 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68004d40 0x7f9d781e3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d78157880 0x7f9d78156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78071bb0 con 0x7f9d78157400 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d780718d0 con 0x7f9d68004d40 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.584+0000 7f9d7ef09640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9d78071e90 con 0x7f9d78157880 2024-05-09T08:56:08.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47940/0 (socket says 172.21.15.49:47940) 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 -- 172.21.15.49:0/3336418372 learned_addr learned my addr 172.21.15.49:0/3336418372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d7d4a6640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d78157880 0x7f9d78156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68004d40 0x7f9d781e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 -- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d78157880 msgr2=0x7f9d78156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d78157880 0x7f9d78156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 -- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68004d40 msgr2=0x7f9d781e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68004d40 0x7f9d781e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 -- 172.21.15.49:0/3336418372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d78155920 con 0x7f9d78157400 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d77fff640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c700 tx=0x7f9d6400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018070 con 0x7f9d78157400 2024-05-09T08:56:08.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d640074e0 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7815af40 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64004a20 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.585+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7815ba80 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.586+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d78068560 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.587+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d6401b070 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.587+0000 7f9d757fa640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 0x7f9d5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.587+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d6408a3b0 con 0x7f9d78157400 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.587+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 0x7f9d5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.588+0000 7f9d7cca5640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 0x7f9d5c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009b10 tx=0x7f9d6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.592+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d780c8350 con 0x7f9d78157400 2024-05-09T08:56:08.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.776+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d780727e0 con 0x7f9d78157400 2024-05-09T08:56:08.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.777+0000 7f9d757fa640 1 -- 172.21.15.49:0/3336418372 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f9d64058d60 con 0x7f9d78157400 2024-05-09T08:56:08.777 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:08.777 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:08.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 msgr2=0x7f9d5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 0x7f9d5c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009b10 tx=0x7f9d6c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:08.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 msgr2=0x7f9d781563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400c700 tx=0x7f9d6400cbc0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 shutdown_connections 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9d5c06be30 0x7f9d5c06e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9d78157880 0x7f9d78156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9d78157400 0x7f9d781563c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 --2- 172.21.15.49:0/3336418372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d68004d40 0x7f9d781e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 >> 172.21.15.49:0/3336418372 conn(0x7f9d78074840 msgr2=0x7f9d780ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 shutdown_connections 2024-05-09T08:56:08.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.780+0000 7f9d7ef09640 1 -- 172.21.15.49:0/3336418372 wait complete. 2024-05-09T08:56:08.790 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T08:56:08.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:08.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.895+0000 7f38e9e38640 1 Processor -- start 2024-05-09T08:56:08.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.895+0000 7f38e9e38640 1 -- start start 2024-05-09T08:56:08.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e9e38640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38e4155340 0x7f38e415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e9e38640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38e40729e0 0x7f38e415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e9e38640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e415b390 con 0x7f38e4154f60 2024-05-09T08:56:08.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e9e38640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f38e415c2d0 con 0x7f38e4155340 2024-05-09T08:56:08.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e9e38640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f38e415b670 con 0x7f38e40729e0 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38daffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38e4155340 0x7f38e415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f38e4154f60 0x7f38e406b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45914/0 (socket says 172.21.15.49:45914) 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e37fe640 1 -- 172.21.15.49:0/3661629139 learned_addr learned my addr 172.21.15.49:0/3661629139 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38daffd640 1 -- 172.21.15.49:0/3661629139 >> v2:172.21.15.183:3300/0 conn(0x7f38e40729e0 msgr2=0x7f38e415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38daffd640 1 --2- 172.21.15.49:0/3661629139 >> v2:172.21.15.183:3300/0 conn(0x7f38e40729e0 0x7f38e415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38daffd640 1 -- 172.21.15.49:0/3661629139 >> v1:172.21.15.49:6789/0 conn(0x7f38e4154f60 legacy=0x7f38e406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38e2ffd640 1 -- 172.21.15.49:0/3661629139 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869214937 0 0) 0x7f38e415b390 con 0x7f38e4154f60 2024-05-09T08:56:08.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.896+0000 7f38daffd640 1 -- 172.21.15.49:0/3661629139 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e415b950 con 0x7f38e4155340 2024-05-09T08:56:08.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38daffd640 1 --2- 172.21.15.49:0/3661629139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e4155340 0x7f38e415ca40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38e406d030 tx=0x7f38cc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09045babd664ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e2ffd640 1 -- 172.21.15.49:0/3661629139 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc037070 con 0x7f38e4155340 2024-05-09T08:56:08.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e2ffd640 1 -- 172.21.15.49:0/3661629139 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc032440 con 0x7f38e4155340 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e2ffd640 1 -- 172.21.15.49:0/3661629139 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc032810 con 0x7f38e4155340 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 -- 172.21.15.49:0/3661629139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e4155340 msgr2=0x7f38e415ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 --2- 172.21.15.49:0/3661629139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e4155340 0x7f38e415ca40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f38e406d030 tx=0x7f38cc02a230 comp rx=0 tx=0).stop 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 -- 172.21.15.49:0/3661629139 shutdown_connections 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 --2- 172.21.15.49:0/3661629139 >> v2:172.21.15.183:3300/0 conn(0x7f38e40729e0 0x7f38e415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 --2- 172.21.15.49:0/3661629139 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e4155340 0x7f38e415ca40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.897+0000 7f38e9e38640 1 -- 172.21.15.49:0/3661629139 >> 172.21.15.49:0/3661629139 conn(0x7f38e4067d90 msgr2=0x7f38e4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.898+0000 7f38e9e38640 1 -- 172.21.15.49:0/3661629139 shutdown_connections 2024-05-09T08:56:08.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.898+0000 7f38e9e38640 1 -- 172.21.15.49:0/3661629139 wait complete. 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.899+0000 7f38e9e38640 1 Processor -- start 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.899+0000 7f38e9e38640 1 -- start start 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.899+0000 7f38e9e38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 0x7f38e40cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.899+0000 7f38e9e38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 0x7f38e414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38e4073210 con 0x7f38e415a430 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38e415c520 con 0x7f38e4154f60 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38e406d110 con 0x7f38e40729e0 2024-05-09T08:56:08.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47958/0 (socket says 172.21.15.49:47958) 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 -- 172.21.15.49:0/1763988627 learned_addr learned my addr 172.21.15.49:0/1763988627 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38daffd640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 0x7f38e414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e37fe640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 0x7f38e40cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 -- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 msgr2=0x7f38e40cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 0x7f38e40cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 -- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 msgr2=0x7f38e414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 0x7f38e414f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 -- 172.21.15.49:0/1763988627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e41f0a40 con 0x7f38e415a430 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e37fe640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 0x7f38e40cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38daffd640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 0x7f38e414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:08.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e3fff640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b970 tx=0x7f38dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc019070 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc00cb60 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc014440 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e41f0ce0 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.900+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e40ca3d0 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.901+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e406bdd0 con 0x7f38e415a430 2024-05-09T08:56:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.902+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38dc01c070 con 0x7f38e415a430 2024-05-09T08:56:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.902+0000 7f38e0ff9640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 0x7f38c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.902+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38dc08aba0 con 0x7f38e415a430 2024-05-09T08:56:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.903+0000 7f38e37fe640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 0x7f38c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:08.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.903+0000 7f38e37fe640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 0x7f38c406e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000f60 tx=0x7f38d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:08.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:08.906+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38dc059550 con 0x7f38e415a430 2024-05-09T08:56:09.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.089+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38e4073520 con 0x7f38e415a430 2024-05-09T08:56:09.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.090+0000 7f38e0ff9640 1 -- 172.21.15.49:0/1763988627 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f38dc058e50 con 0x7f38e415a430 2024-05-09T08:56:09.089 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:09.089 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 msgr2=0x7f38c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 0x7f38c406e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000f60 tx=0x7f38d4009040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 msgr2=0x7f38e414f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b970 tx=0x7f38dc00be30 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 shutdown_connections 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38c406be60 0x7f38c406e320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38e415a430 0x7f38e414f6c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38e4154f60 0x7f38e414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 --2- 172.21.15.49:0/1763988627 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38e40729e0 0x7f38e40cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 >> 172.21.15.49:0/1763988627 conn(0x7f38e4067d90 msgr2=0x7f38e415d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 shutdown_connections 2024-05-09T08:56:09.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.092+0000 7f38e9e38640 1 -- 172.21.15.49:0/1763988627 wait complete. 2024-05-09T08:56:09.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T08:56:09.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T08:56:09.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T08:56:09.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T08:56:09.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T08:56:09.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T08:56:09.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T08:56:09.106 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T08:56:09.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.283+0000 7f1e09b7e640 1 Processor -- start 2024-05-09T08:56:09.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.284+0000 7f1e09b7e640 1 -- start start 2024-05-09T08:56:09.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f32fe5c8640 1 Processor -- start 2024-05-09T08:56:09.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e09b7e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1dfc0462f0 0x7f1dfc007d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e09b7e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc04bc20 0x7f1dfc0470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e09b7e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dfc04c090 con 0x7f1dfc0dd6b0 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e09b7e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc0dda90 con 0x7f1dfc0462f0 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e09b7e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0078d0 con 0x7f1dfc04bc20 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e02ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1dfc0462f0 0x7f1dfc007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e02ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1dfc0462f0 0x7f1dfc007d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47968/0 (socket says 172.21.15.49:47968) 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f1e02ffd640 1 -- 172.21.15.49:0/1532256747 learned_addr learned my addr 172.21.15.49:0/1532256747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.286+0000 7f1e02ffd640 1 -- 172.21.15.49:0/1532256747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc04bc20 msgr2=0x7f1dfc0470a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.286+0000 7f1e02ffd640 1 --2- 172.21.15.49:0/1532256747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc04bc20 0x7f1dfc0470a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.286+0000 7f1e02ffd640 1 -- 172.21.15.49:0/1532256747 >> v1:172.21.15.183:6789/0 conn(0x7f1dfc0dd6b0 legacy=0x7f1dfc047be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:09.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.286+0000 7f1e02ffd640 1 -- 172.21.15.49:0/1532256747 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc006fe0 con 0x7f1dfc0462f0 2024-05-09T08:56:09.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.285+0000 7f32fe5c8640 1 -- start start 2024-05-09T08:56:09.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f6edaed4640 1 Processor -- start 2024-05-09T08:56:09.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f6edaed4640 1 -- start start 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fe5c8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f32f0050c90 0x7f32f00dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fe5c8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32f00062a0 0x7f32f0034a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fe5c8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f0007700 con 0x7f32f0005e80 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fe5c8640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f32f00068a0 con 0x7f32f00062a0 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fe5c8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f32f0006b80 con 0x7f32f0050c90 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32f77fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f32f0005e80 0x7f32f004b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45918/0 (socket says 172.21.15.49:45918) 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32f77fe640 1 -- 172.21.15.49:0/2938080816 learned_addr learned my addr 172.21.15.49:0/2938080816 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f32fcb65640 1 --2- 172.21.15.49:0/2938080816 >> v2:172.21.15.49:3300/0 conn(0x7f32f00062a0 0x7f32f0034a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f7c8a638640 1 Processor -- start 2024-05-09T08:56:09.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.287+0000 7f7c8a638640 1 -- start start 2024-05-09T08:56:09.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f9722ddd640 1 Processor -- start 2024-05-09T08:56:09.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f9722ddd640 1 -- start start 2024-05-09T08:56:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f6edaed4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6ed40cba60 0x7f6ed415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f32f7fff640 1 --2- 172.21.15.49:0/2938080816 >> v2:172.21.15.183:3300/0 conn(0x7f32f0050c90 0x7f32f00dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f6edaed4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed415e410 con 0x7f6ed4155190 2024-05-09T08:56:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f6edaed4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed415ae60 con 0x7f6ed40b3530 2024-05-09T08:56:09.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f6edaed4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed415b120 con 0x7f6ed40cba60 2024-05-09T08:56:09.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f32f7fff640 1 -- 172.21.15.49:0/2938080816 >> v2:172.21.15.49:3300/0 conn(0x7f32f00062a0 msgr2=0x7f32f0034a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f32f7fff640 1 --2- 172.21.15.49:0/2938080816 >> v2:172.21.15.49:3300/0 conn(0x7f32f00062a0 0x7f32f0034a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f32f7fff640 1 -- 172.21.15.49:0/2938080816 >> v1:172.21.15.49:6789/0 conn(0x7f32f0005e80 legacy=0x7f32f004b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f32f7fff640 1 -- 172.21.15.49:0/2938080816 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f0006e60 con 0x7f32f0050c90 2024-05-09T08:56:09.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f5d9759e640 1 Processor -- start 2024-05-09T08:56:09.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c8a638640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7c84074e90 0x7f7c840c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f5d9759e640 1 -- start start 2024-05-09T08:56:09.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f6ed3fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6ed40b3530 0x7f6ed40cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36352/0 (socket says 172.21.15.49:36352) 2024-05-09T08:56:09.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f6ed3fff640 1 -- 172.21.15.49:0/134425315 learned_addr learned my addr 172.21.15.49:0/134425315 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c8a638640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c840c9570 con 0x7f7c84074ab0 2024-05-09T08:56:09.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c837fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7c84074e90 0x7f7c840c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c837fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7c84074e90 0x7f7c840c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55318/0 (socket says 172.21.15.49:55318) 2024-05-09T08:56:09.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c837fe640 1 -- 172.21.15.49:0/1527937892 learned_addr learned my addr 172.21.15.49:0/1527937892 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c840c9850 con 0x7f7c84073330 2024-05-09T08:56:09.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7c840c8e00 con 0x7f7c84074e90 2024-05-09T08:56:09.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f32f7fff640 1 --2- 172.21.15.49:0/2938080816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f0050c90 0x7f32f00dac50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32f004a7b0 tx=0x7f32e402a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1539b5f8ab091c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f689fba9640 1 Processor -- start 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2444146106 0 0) 0x7f6ed415ae60 con 0x7f6ed40b3530 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ebc003520 con 0x7f6ed40b3530 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.288+0000 7f1e02ffd640 1 --2- 172.21.15.49:0/1532256747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0462f0 0x7f1dfc007d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009870 tx=0x7f1df002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8cdfaaa12f9595 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f1e027fc640 1 -- 172.21.15.49:0/1532256747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037070 con 0x7f1dfc0462f0 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f1e027fc640 1 -- 172.21.15.49:0/1532256747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df0032440 con 0x7f1dfc0462f0 2024-05-09T08:56:09.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.289+0000 7f1e027fc640 1 -- 172.21.15.49:0/1532256747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0032810 con 0x7f1dfc0462f0 2024-05-09T08:56:09.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f9722ddd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97140e2500 0x7f9714046860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed9471640 1 --2- 172.21.15.49:0/134425315 >> v2:172.21.15.49:3300/0 conn(0x7f6ed40cba60 0x7f6ed415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f9722ddd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971404b3f0 con 0x7f97140e28e0 2024-05-09T08:56:09.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f9722ddd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9714045aa0 con 0x7f97140e2e00 2024-05-09T08:56:09.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f9722ddd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9714045850 con 0x7f97140e2500 2024-05-09T08:56:09.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f689fba9640 1 -- start start 2024-05-09T08:56:09.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f32f6ffd640 1 -- 172.21.15.49:0/2938080816 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4037070 con 0x7f32f0050c90 2024-05-09T08:56:09.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f32f6ffd640 1 -- 172.21.15.49:0/2938080816 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e4032440 con 0x7f32f0050c90 2024-05-09T08:56:09.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f32f6ffd640 1 -- 172.21.15.49:0/2938080816 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4037070 con 0x7f32f0050c90 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3147805381 0 0) 0x7f6ebc003520 con 0x7f6ed40b3530 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 >> v2:172.21.15.49:3300/0 conn(0x7f6ed40cba60 msgr2=0x7f6ed415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 --2- 172.21.15.49:0/134425315 >> v2:172.21.15.49:3300/0 conn(0x7f6ed40cba60 0x7f6ed415e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 >> v1:172.21.15.49:6789/0 conn(0x7f6ed4155190 legacy=0x7f6ed40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed415b400 con 0x7f6ed40b3530 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f6ec8001670 con 0x7f6ed40b3530 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 >> v1:172.21.15.183:6789/0 conn(0x7f6ed40b3530 legacy=0x7f6ed40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ebc0039b0 0x7f6ebc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ebc004800 0x7f6ebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ebc0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed415b120 con 0x7f6ebc004800 2024-05-09T08:56:09.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f97215da640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f97140e2e00 0x7f971404c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36376/0 (socket says 172.21.15.49:36376) 2024-05-09T08:56:09.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc014900 con 0x7f6ebc0051c0 2024-05-09T08:56:09.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6ebc014b00 con 0x7f6ebc0039b0 2024-05-09T08:56:09.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f97215da640 1 -- 172.21.15.49:0/1207319398 learned_addr learned my addr 172.21.15.49:0/1207319398 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f9721ddb640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97140e2500 0x7f9714046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f5d9759e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d98159370 0x7f5d9806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761362411 0 0) 0x7f7c840c9570 con 0x7f7c84074ab0 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f5d9759e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d9806cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9759e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d9806d480 0x7f5d9815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9759e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5d9806e200 con 0x7f5d9806d480 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9759e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5d98163120 con 0x7f5d98159370 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9759e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5d98159750 con 0x7f5d980c7480 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d98159370 0x7f5d9806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d98159370 0x7f5d9806dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55330/0 (socket says 172.21.15.49:55330) 2024-05-09T08:56:09.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 -- 172.21.15.49:0/662618811 learned_addr learned my addr 172.21.15.49:0/662618811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ebc0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.290+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c6c003520 con 0x7f7c84074ab0 2024-05-09T08:56:09.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296946915 0 0) 0x7f7c840c9850 con 0x7f7c84073330 2024-05-09T08:56:09.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c840c9570 con 0x7f7c84073330 2024-05-09T08:56:09.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed3fff640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ebc004800 0x7f6ebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d8dd9b640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d9806cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1036209993 0 0) 0x7f7c840c9570 con 0x7f7c84073330 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 -- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 msgr2=0x7f5d9806cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d9806cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 -- 172.21.15.49:0/662618811 >> v2:172.21.15.49:3300/0 conn(0x7f5d9806d480 msgr2=0x7f5d9815efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 --2- 172.21.15.49:0/662618811 >> v2:172.21.15.49:3300/0 conn(0x7f5d9806d480 0x7f5d9815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f689fba9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6898151a60 0x7f6898071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f5d9659c640 1 -- 172.21.15.49:0/662618811 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980ca760 con 0x7f5d98159370 2024-05-09T08:56:09.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 >> v2:172.21.15.183:3300/0 conn(0x7f7c84074e90 msgr2=0x7f7c840c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 --2- 172.21.15.49:0/1527937892 >> v2:172.21.15.183:3300/0 conn(0x7f7c84074e90 0x7f7c840c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 >> v1:172.21.15.49:6789/0 conn(0x7f7c84074ab0 legacy=0x7f7c8415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c840711e0 con 0x7f7c84073330 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f689d945640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6898151a60 0x7f6898071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f689e146640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6898157790 0x7f68980cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45956/0 (socket says 172.21.15.49:45956) 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689d945640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6898151a60 0x7f6898071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55350/0 (socket says 172.21.15.49:55350) 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689e146640 1 -- 172.21.15.49:0/1189818846 learned_addr learned my addr 172.21.15.49:0/1189818846 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2938080816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f0050c90 msgr2=0x7f32f00dac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 --2- 172.21.15.49:0/2938080816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f0050c90 0x7f32f00dac50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32f004a7b0 tx=0x7f32e402a1f0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f7c78001670 con 0x7f7c84073330 2024-05-09T08:56:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6898071010 con 0x7f6898157790 2024-05-09T08:56:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 >> v1:172.21.15.183:6789/0 conn(0x7f7c84073330 legacy=0x7f7c84154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2938080816 shutdown_connections 2024-05-09T08:56:09.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 --2- 172.21.15.49:0/2938080816 >> v2:172.21.15.49:3300/0 conn(0x7f32f00062a0 0x7f32f0034a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 --2- 172.21.15.49:0/2938080816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f0050c90 0x7f32f00dac50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2938080816 >> 172.21.15.49:0/2938080816 conn(0x7f32f0050170 msgr2=0x7f32f0050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2521825170 0 0) 0x7f9714045aa0 con 0x7f97140e2e00 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9708003520 con 0x7f97140e2e00 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1865077158 0 0) 0x7f9708003520 con 0x7f97140e2e00 2024-05-09T08:56:09.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2938080816 shutdown_connections 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97140e2500 msgr2=0x7f9714046860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97140e2500 0x7f9714046860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 >> v1:172.21.15.49:6789/0 conn(0x7f97140e28e0 legacy=0x7f97140d8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9714045d80 con 0x7f97140e2e00 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f970c001670 con 0x7f97140e2e00 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 >> v1:172.21.15.183:6789/0 conn(0x7f97140e2e00 legacy=0x7f971404c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68980c8ae0 con 0x7f68980c8510 2024-05-09T08:56:09.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f68980c7410 con 0x7f6898151a60 2024-05-09T08:56:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c6c004ba0 0x7f7c6c004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 -- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ebc0039b0 msgr2=0x7f6ebc003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:56:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ebc0039b0 0x7f6ebc003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778512818 0 0) 0x7f68980c8ae0 con 0x7f68980c8510 2024-05-09T08:56:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 -- 172.21.15.49:0/134425315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ebc004800 msgr2=0x7f6ebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ebc004800 0x7f6ebc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f6ed9471640 1 -- 172.21.15.49:0/134425315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed415b400 con 0x7f6ebc0051c0 2024-05-09T08:56:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2938080816 wait complete. 2024-05-09T08:56:09.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6884003520 con 0x7f68980c8510 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97080039e0 0x7f9708003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9708004800 0x7f9708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f970800b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9714045850 con 0x7f9708004800 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f970800bf50 con 0x7f97080039e0 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f970800c150 con 0x7f97080051c0 2024-05-09T08:56:09.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766235699 0 0) 0x7f6898071010 con 0x7f6898157790 2024-05-09T08:56:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f5d9659c640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 0x7f5d9806dcc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d980687b0 tx=0x7f5d8002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2722b892147cbba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68980c8ae0 con 0x7f6898157790 2024-05-09T08:56:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f6ed9471640 1 -- 172.21.15.49:0/134425315 reap_dead start 2024-05-09T08:56:09.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f97225dc640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f970800b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7fccbd77a640 1 Processor -- start 2024-05-09T08:56:09.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c0037a0 0x7f7c6c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1118716049 0 0) 0x7f6884003520 con 0x7f68980c8510 2024-05-09T08:56:09.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 >> v2:172.21.15.183:3300/0 conn(0x7f6898151a60 msgr2=0x7f6898071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 --2- 172.21.15.49:0/1189818846 >> v2:172.21.15.183:3300/0 conn(0x7f6898151a60 0x7f6898071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f5d95d9b640 1 -- 172.21.15.49:0/662618811 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80037070 con 0x7f5d98159370 2024-05-09T08:56:09.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 >> v1:172.21.15.49:6789/0 conn(0x7f6898157790 legacy=0x7f68980cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f5d95d9b640 1 -- 172.21.15.49:0/662618811 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80032440 con 0x7f5d98159370 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f5d95d9b640 1 -- 172.21.15.49:0/662618811 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80032810 con 0x7f5d98159370 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 0x7f7c6c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c840c8e00 con 0x7f7c6c004ba0 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c014850 con 0x7f7c6c004160 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c014a50 con 0x7f7c6c0037a0 2024-05-09T08:56:09.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.291+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 0x7f7c6c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980c8dc0 con 0x7f68980c8510 2024-05-09T08:56:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f6888001670 con 0x7f68980c8510 2024-05-09T08:56:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c837fe640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c0037a0 0x7f7c6c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c88bd5640 1 -- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c0037a0 msgr2=0x7f7c6c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c0037a0 0x7f7c6c003c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c88bd5640 1 -- 172.21.15.49:0/1527937892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c6c004ba0 msgr2=0x7f7c6c004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c6c004ba0 0x7f7c6c004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.292+0000 7f7c88bd5640 1 -- 172.21.15.49:0/1527937892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c840711e0 con 0x7f7c6c004160 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 >> v1:172.21.15.183:6789/0 conn(0x7f68980c8510 legacy=0x7f689806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7fccbd77a640 1 -- start start 2024-05-09T08:56:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884003b40 0x7f6884003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 0x7f6884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f7c88bd5640 1 -- 172.21.15.49:0/1527937892 reap_dead start 2024-05-09T08:56:09.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68840051c0 0x7f688400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689d945640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884003b40 0x7f6884003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689d144640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 0x7f6884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68980c7410 con 0x7f6884003b40 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f688400bf80 con 0x7f68840051c0 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f5d9759e640 1 -- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 msgr2=0x7f5d9806dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f5d9759e640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 0x7f5d9806dcc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d980687b0 tx=0x7f5d8002a210 comp rx=0 tx=0).stop 2024-05-09T08:56:09.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f689e146640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68840051c0 0x7f688400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f688400c180 con 0x7f6884004800 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1532256747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0462f0 msgr2=0x7f1dfc007d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 --2- 172.21.15.49:0/1532256747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0462f0 0x7f1dfc007d60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009870 tx=0x7f1df002a230 comp rx=0 tx=0).stop 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 -- 172.21.15.49:0/662618811 shutdown_connections 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 --2- 172.21.15.49:0/662618811 >> v2:172.21.15.49:3300/0 conn(0x7f5d9806d480 0x7f5d9815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d9806cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 --2- 172.21.15.49:0/662618811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 0x7f5d9806dcc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 -- 172.21.15.49:0/662618811 >> 172.21.15.49:0/662618811 conn(0x7f5d9806b200 msgr2=0x7f5d9806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 0x7f7c6c0141d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c190 tx=0x7f7c6c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb6cc8c4d85d3da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f9721ddb640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97080039e0 0x7f9708003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f97225dc640 1 -- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97080039e0 msgr2=0x7f9708003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f97225dc640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97080039e0 0x7f9708003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1532256747 shutdown_connections 2024-05-09T08:56:09.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 --2- 172.21.15.49:0/1532256747 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc04bc20 0x7f1dfc0470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 --2- 172.21.15.49:0/1532256747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0462f0 0x7f1dfc007d60 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009870 tx=0x7f1df002a230 comp rx=0 tx=0).stop 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1532256747 >> 172.21.15.49:0/1532256747 conn(0x7f1dfc04dd20 msgr2=0x7f1dfc04e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f97225dc640 1 -- 172.21.15.49:0/1207319398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9708004800 msgr2=0x7f9708004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f97225dc640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9708004800 0x7f9708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.293+0000 7f97225dc640 1 -- 172.21.15.49:0/1207319398 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708003520 con 0x7f97080051c0 2024-05-09T08:56:09.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 -- 172.21.15.49:0/662618811 shutdown_connections 2024-05-09T08:56:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f6ed9471640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ebc0141d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007330 tx=0x7f6ebc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c830df7e6081b3e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78007c40 con 0x7f7c6c004160 2024-05-09T08:56:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f32fe5c8640 1 Processor -- start 2024-05-09T08:56:09.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f5d9759e640 1 -- 172.21.15.49:0/662618811 wait complete. 2024-05-09T08:56:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc009be0 con 0x7f6ebc0051c0 2024-05-09T08:56:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc003b90 con 0x7f6ebc0051c0 2024-05-09T08:56:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78031620 con 0x7f7c6c004160 2024-05-09T08:56:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f7c82ffd640 1 -- 172.21.15.49:0/1527937892 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78031a10 con 0x7f7c6c004160 2024-05-09T08:56:09.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7fccbd77a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 0x7fccb80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f32fe5c8640 1 -- start start 2024-05-09T08:56:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7fccbd77a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fccb80cc450 0x7fccb80cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7fccbd77a640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccb815b500 con 0x7fccb8154f00 2024-05-09T08:56:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7fccbd77a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fccb80cd000 con 0x7fccb80cc450 2024-05-09T08:56:09.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7fccbd77a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fccb815b7e0 con 0x7fccb80cc000 2024-05-09T08:56:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.294+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1532256747 shutdown_connections 2024-05-09T08:56:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 0x7fccb80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 0x7fccb80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34166/0 (socket says 172.21.15.49:34166) 2024-05-09T08:56:09.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7fccb77fe640 1 -- 172.21.15.49:0/1311602195 learned_addr learned my addr 172.21.15.49:0/1311602195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689d144640 1 -- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68840051c0 msgr2=0x7f688400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689d144640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68840051c0 0x7f688400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689d144640 1 -- 172.21.15.49:0/1189818846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884003b40 msgr2=0x7f6884003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689d144640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884003b40 0x7f6884003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689d144640 1 -- 172.21.15.49:0/1189818846 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6884003520 con 0x7f6884004800 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f689e146640 1 -- 172.21.15.49:0/1189818846 reap_dead start 2024-05-09T08:56:09.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f6ed37fe640 1 -- 172.21.15.49:0/134425315 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004020 con 0x7f6ebc0051c0 2024-05-09T08:56:09.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1311602195 >> v2:172.21.15.183:3300/0 conn(0x7fccb80cc450 0x7fccb80cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32f0005e80 0x7f32f00cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f00062a0 0x7f32f0178670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 0x7f32f0178bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32f004bc00 con 0x7f32f0050c90 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32f0007420 con 0x7f32f00062a0 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32fe5c8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f32f004bd80 con 0x7f32f0005e80 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f32f7fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32f0005e80 0x7f32f00cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32f77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f00062a0 0x7f32f0178670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 0x7f32f0178bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 0x7f32f0178bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48026/0 (socket says 172.21.15.49:48026) 2024-05-09T08:56:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f689d144640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 0x7f6884004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68880037f0 tx=0x7f6884005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb345635e57bd2b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 msgr2=0x7f7c6c0141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f7c8a638640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 0x7f7c6c0141d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c190 tx=0x7f7c6c005830 comp rx=0 tx=0).stop 2024-05-09T08:56:09.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccb6ffd640 1 -- 172.21.15.49:0/1311602195 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1353424134 0 0) 0x7fccb815b500 con 0x7fccb8154f00 2024-05-09T08:56:09.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccb6ffd640 1 -- 172.21.15.49:0/1311602195 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcca0003520 con 0x7fccb8154f00 2024-05-09T08:56:09.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 -- 172.21.15.49:0/2308747225 learned_addr learned my addr 172.21.15.49:0/2308747225 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f6edaed4640 1 -- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 msgr2=0x7f6ebc0141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f6edaed4640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ebc0141d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc007330 tx=0x7f6ebc005830 comp rx=0 tx=0).stop 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccbcf79640 1 -- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 msgr2=0x7fccb80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 0x7fccb80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 shutdown_connections 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f7c8a638640 1 --2- 172.21.15.49:0/1527937892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c6c004160 0x7f7c6c0141d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 >> 172.21.15.49:0/1527937892 conn(0x7f7c8406b200 msgr2=0x7f7c8406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccbcf79640 1 -- 172.21.15.49:0/1311602195 >> v1:172.21.15.49:6789/0 conn(0x7fccb8154f00 legacy=0x7fccb815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6888003830 con 0x7f6884004800 2024-05-09T08:56:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6888033070 con 0x7f6884004800 2024-05-09T08:56:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f5d9759e640 1 Processor -- start 2024-05-09T08:56:09.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7fccbcf79640 1 -- 172.21.15.49:0/1311602195 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb815bac0 con 0x7fccb80cc450 2024-05-09T08:56:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 -- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32f0005e80 msgr2=0x7f32f00cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f689c943640 1 -- 172.21.15.49:0/1189818846 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688802f980 con 0x7f6884004800 2024-05-09T08:56:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f5d9759e640 1 -- start start 2024-05-09T08:56:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f97225dc640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f970800b930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f971c050dd0 tx=0x7f9708005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ddeca4e242428f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32f0005e80 0x7f32f00cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 -- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f00062a0 msgr2=0x7f32f0178670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32f00062a0 0x7f32f0178670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00d7680 con 0x7f32f0050c90 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.295+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1532256747 wait complete. 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f1e09b7e640 1 Processor -- start 2024-05-09T08:56:09.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.296+0000 7f32fcb65640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 0x7f32f0178bb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00eb40 tx=0x7f32ec00c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 shutdown_connections 2024-05-09T08:56:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f6edaed4640 1 -- 172.21.15.49:0/134425315 shutdown_connections 2024-05-09T08:56:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f6edaed4640 1 --2- 172.21.15.49:0/134425315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ebc0141d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f6edaed4640 1 -- 172.21.15.49:0/134425315 >> 172.21.15.49:0/134425315 conn(0x7f6ed4074a20 msgr2=0x7f6ed4074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f6edaed4640 1 -- 172.21.15.49:0/134425315 shutdown_connections 2024-05-09T08:56:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f6edaed4640 1 -- 172.21.15.49:0/134425315 wait complete. 2024-05-09T08:56:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f1e09b7e640 1 -- start start 2024-05-09T08:56:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0af070 con 0x7f97080051c0 2024-05-09T08:56:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c0ab6d0 con 0x7f97080051c0 2024-05-09T08:56:09.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f9720dd9640 1 -- 172.21.15.49:0/1207319398 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c04fb60 con 0x7f97080051c0 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 msgr2=0x7f6884004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 0x7f6884004c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68880037f0 tx=0x7f6884005830 comp rx=0 tx=0).stop 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 shutdown_connections 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 --2- 172.21.15.49:0/1189818846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6884004800 0x7f6884004c80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 >> 172.21.15.49:0/1189818846 conn(0x7f689806ef50 msgr2=0x7f689806f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 shutdown_connections 2024-05-09T08:56:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f689fba9640 1 -- 172.21.15.49:0/1189818846 wait complete. 2024-05-09T08:56:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec017070 con 0x7f32f0050c90 2024-05-09T08:56:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f5d9759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 0x7f5d981f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.297+0000 7f7c8a638640 1 -- 172.21.15.49:0/1527937892 wait complete. 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f00d7960 con 0x7f32f0050c90 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f00d7f20 con 0x7f32f0050c90 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ec00c980 con 0x7f32f0050c90 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00cd50 con 0x7f32f0050c90 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f1e09b7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc04bc20 0x7f1dfc0e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f9722ddd640 1 -- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 msgr2=0x7f970800b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f9722ddd640 1 --2- 172.21.15.49:0/1207319398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f970800b930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f971c050dd0 tx=0x7f9708005830 comp rx=0 tx=0).stop 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f97225dc640 1 -- 172.21.15.49:0/1207319398 reap_dead start 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f1e09b7e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0dd6b0 0x7f1dfc0e4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f1e037fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc04bc20 0x7f1dfc0e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f1e037fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc04bc20 0x7f1dfc0e0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55392/0 (socket says 172.21.15.49:55392) 2024-05-09T08:56:09.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f1e037fe640 1 -- 172.21.15.49:0/1670043785 learned_addr learned my addr 172.21.15.49:0/1670043785 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f6edaed4640 1 Processor -- start 2024-05-09T08:56:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f1e09b7e640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc0e7000 0x7f1dfc0e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f6edaed4640 1 -- start start 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9759e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d981556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9759e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 0x7f5d9815e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9759e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d981627f0 con 0x7f5d9806d480 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9759e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98162510 con 0x7f5d98159370 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9759e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5d98162ad0 con 0x7f5d980c7480 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 0x7f5d981f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 0x7f5d981f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48042/0 (socket says 172.21.15.49:48042) 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.298+0000 7f5d9659c640 1 -- 172.21.15.49:0/4056398133 learned_addr learned my addr 172.21.15.49:0/4056398133 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc007560 con 0x7f1dfc0dd6b0 2024-05-09T08:56:09.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32d8001030 con 0x7f32f0050c90 2024-05-09T08:56:09.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f9722ddd640 1 -- 172.21.15.49:0/1207319398 shutdown_connections 2024-05-09T08:56:09.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f9722ddd640 1 -- 172.21.15.49:0/1207319398 >> 172.21.15.49:0/1207319398 conn(0x7f9714050170 msgr2=0x7f9714050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f1e03fff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc0e7000 0x7f1dfc0e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc007280 con 0x7f1dfc04bc20 2024-05-09T08:56:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f689fba9640 1 Processor -- start 2024-05-09T08:56:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0e4180 con 0x7f1dfc0e7000 2024-05-09T08:56:09.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f5d9659c640 1 -- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 msgr2=0x7f5d981556e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f5d9659c640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d980c7480 0x7f5d981556e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f5d9659c640 1 -- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 msgr2=0x7f5d9815e750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f5d9659c640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d98159370 0x7f5d9815e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f5d9659c640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980cb240 con 0x7f5d9806d480 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f9722ddd640 1 -- 172.21.15.49:0/1207319398 shutdown_connections 2024-05-09T08:56:09.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f689fba9640 1 -- start start 2024-05-09T08:56:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb80cb670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbca7b25858f2ade server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ed4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f5d9659c640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 0x7f5d981f4c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d800097f0 tx=0x7f5d800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7fccb6ffd640 1 -- 172.21.15.49:0/1311602195 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007740 con 0x7fccb80cc450 2024-05-09T08:56:09.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed415dd90 0x7f6ed40cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7fccb6ffd640 1 -- 172.21.15.49:0/1311602195 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb00078e0 con 0x7fccb80cc450 2024-05-09T08:56:09.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 0x7f6ed41efae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed415bca0 con 0x7f6ed415dd90 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed415b9c0 con 0x7f6ebc0051c0 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6edaed4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4155570 con 0x7f6ed41538d0 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f1e03fff640 1 -- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc04bc20 msgr2=0x7f1dfc0e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f1e03fff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc04bc20 0x7f1dfc0e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f1e03fff640 1 -- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0dd6b0 msgr2=0x7f1dfc0e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f1e03fff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dfc0dd6b0 0x7f1dfc0e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80041020 con 0x7f5d9806d480 2024-05-09T08:56:09.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6ed9471640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 0x7f6ed41efae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6ed9471640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 0x7f6ed41efae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34192/0 (socket says 172.21.15.49:34192) 2024-05-09T08:56:09.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f6ed9471640 1 -- 172.21.15.49:0/4114375728 learned_addr learned my addr 172.21.15.49:0/4114375728 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80030a40 con 0x7f5d9806d480 2024-05-09T08:56:09.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f5d9759e640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d980cb520 con 0x7f5d9806d480 2024-05-09T08:56:09.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32ec01a070 con 0x7f32f0050c90 2024-05-09T08:56:09.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884004800 0x7f68981e3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68980cde20 0x7f68980cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 0x7f6898149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6898157c20 con 0x7f6884004800 2024-05-09T08:56:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.299+0000 7f9722ddd640 1 -- 172.21.15.49:0/1207319398 wait complete. 2024-05-09T08:56:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6898071560 con 0x7f6898156960 2024-05-09T08:56:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689fba9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6898151e40 con 0x7f68980cde20 2024-05-09T08:56:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689e146640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 0x7f6898149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccbd77a640 1 -- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 msgr2=0x7fccb80cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689e146640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 0x7f6898149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55416/0 (socket says 172.21.15.49:55416) 2024-05-09T08:56:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f689e146640 1 -- 172.21.15.49:0/2569067692 learned_addr learned my addr 172.21.15.49:0/2569067692 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f6ed9471640 1 -- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 msgr2=0x7f6ed4154090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:09.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f32f4ff9640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f32dc06beb0 0x7f32dc06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccb6ffd640 1 -- 172.21.15.49:0/1311602195 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0007740 con 0x7fccb80cc450 2024-05-09T08:56:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccbd77a640 1 --2- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb80cb670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccbd77a640 1 -- 172.21.15.49:0/1311602195 shutdown_connections 2024-05-09T08:56:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccbd77a640 1 --2- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb80cb670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7fccbd77a640 1 --2- 172.21.15.49:0/1311602195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb80cc000 0x7fccb80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f7c8a638640 1 Processor -- start 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.300+0000 7f7c8a638640 1 -- start start 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c004160 0x7f7c84157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84155c50 0x7f7c841451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c84156d20 0x7f7c841432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7fccbd77a640 1 -- 172.21.15.49:0/1311602195 >> 172.21.15.49:0/1311602195 conn(0x7fccb8067760 msgr2=0x7fccb8072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c840c9b30 con 0x7f7c84156d20 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c840717a0 con 0x7f7c84155c50 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f7c8a638640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c84071cc0 con 0x7f7c6c004160 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c88bd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c84156d20 0x7f7c841432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c88bd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c84156d20 0x7f7c841432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48070/0 (socket says 172.21.15.49:48070) 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c88bd5640 1 -- 172.21.15.49:0/4009354850 learned_addr learned my addr 172.21.15.49:0/4009354850 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32ec08b080 con 0x7f32f0050c90 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f1e03fff640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc0d52b0 con 0x7f1dfc0e7000 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c004160 0x7f7c84157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f6ed9471640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ebc0051c0 0x7f6ed4154090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f6ed9471640 1 -- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed415dd90 msgr2=0x7f6ed40cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f6ed9471640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6ed415dd90 0x7f6ed40cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f5d9759e640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d98153a90 con 0x7f5d9806d480 2024-05-09T08:56:09.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80037040 con 0x7f5d9806d480 2024-05-09T08:56:09.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7fccbd77a640 1 -- 172.21.15.49:0/1311602195 shutdown_connections 2024-05-09T08:56:09.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f32f4ff9640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f32dc0740b0 0x7f32dc0764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f32f7fff640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f32dc06beb0 0x7f32dc06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32ec016ec0 con 0x7f32dc0740b0 2024-05-09T08:56:09.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7fccbd77a640 1 -- 172.21.15.49:0/1311602195 wait complete. 2024-05-09T08:56:09.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f6ed9471640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed41ed500 con 0x7f6ed41538d0 2024-05-09T08:56:09.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f1e03fff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc0e7000 0x7f1dfc0e4940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b410 tx=0x7f1df800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c837fe640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84155c50 0x7f7c841451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f5d9759e640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d7c001030 con 0x7f5d9806d480 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 -- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84155c50 msgr2=0x7f7c841451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c84155c50 0x7f7c841451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 -- 172.21.15.49:0/4009354850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c84156d20 msgr2=0x7f7c841432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c84156d20 0x7f7c841432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f6ed9471640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 0x7f6ed41efae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc039d40 tx=0x7f6ecc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f7c83fff640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c840cd210 con 0x7f7c6c004160 2024-05-09T08:56:09.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f9722ddd640 1 Processor -- start 2024-05-09T08:56:09.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.301+0000 7f9722ddd640 1 -- start start 2024-05-09T08:56:09.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f689d945640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884004800 0x7f68981e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f32ec092050 con 0x7f32f0050c90 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02e630 con 0x7f6ed41538d0 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f32f7fff640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f32dc06beb0 0x7f32dc06e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32e4034000 tx=0x7f32e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f32f77fe640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f32dc0740b0 0x7f32dc0764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8015020 con 0x7f1dfc0e7000 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc0d5520 con 0x7f1dfc0e7000 2024-05-09T08:56:09.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f1e09b7e640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc0d5ae0 con 0x7f1dfc0e7000 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f6edaed4640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed41ed220 con 0x7f6ed41538d0 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f6edaed4640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed4152b20 con 0x7f6ed41538d0 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc02e7f0 con 0x7f6ed41538d0 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc02ebe0 con 0x7f6ed41538d0 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d80044020 con 0x7f5d9806d480 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f97141822c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 0x7f97140e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9714172b40 0x7f9714031fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f97140d9210 con 0x7f9714172b40 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9714046340 con 0x7f9714031460 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f9722ddd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f97140d9390 con 0x7f97080051c0 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f7c83fff640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c004160 0x7f7c84157510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00c8d0 tx=0x7f7c7c00cd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c003b90 con 0x7f7c6c004160 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c7c003d30 con 0x7f7c6c004160 2024-05-09T08:56:09.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00aa90 con 0x7f7c6c004160 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.302+0000 7f689d144640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68980cde20 0x7f68980cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689e146640 1 -- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68980cde20 msgr2=0x7f68980cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689e146640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68980cde20 0x7f68980cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689e146640 1 -- 172.21.15.49:0/2569067692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884004800 msgr2=0x7f68981e3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689e146640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884004800 0x7f68981e3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689e146640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68980cc460 con 0x7f6898156960 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.303+0000 7f689d945640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6884004800 0x7f68981e3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c84156190 con 0x7f7c6c004160 2024-05-09T08:56:09.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c84156750 con 0x7f7c6c004160 2024-05-09T08:56:09.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f97215da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 0x7f97140e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f689e146640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 0x7f6898149ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6898072bd0 tx=0x7f689000f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f97215da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 0x7f97140e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55430/0 (socket says 172.21.15.49:55430) 2024-05-09T08:56:09.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f97215da640 1 -- 172.21.15.49:0/1359626007 learned_addr learned my addr 172.21.15.49:0/1359626007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7fccbd77a640 1 Processor -- start 2024-05-09T08:56:09.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8011830 con 0x7f1dfc0e7000 2024-05-09T08:56:09.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f32f77fe640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f32dc0740b0 0x7f32dc0764d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32ec016ec0 con 0x7f32dc0740b0 2024-05-09T08:56:09.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d7806bf30 0x7f5d7806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5d800aba70 con 0x7f5d9806d480 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d8f7fe640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d780741b0 0x7f5d780765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d80032440 con 0x7f5d780741b0 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5d800b02a0 con 0x7f5d9806d480 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d8dd9b640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d7806bf30 0x7f5d7806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d96d9d640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d780741b0 0x7f5d780765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f9721ddb640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f97141822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 -- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 msgr2=0x7f97141822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f97080051c0 0x7f97141822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 -- 172.21.15.49:0/1359626007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9714172b40 msgr2=0x7f9714031fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9714172b40 0x7f9714031fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97140e0000 con 0x7f9714031460 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7fccbd77a640 1 -- start start 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689001a070 con 0x7f6898156960 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689000fba0 con 0x7f6898156960 2024-05-09T08:56:09.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890012350 con 0x7f6898156960 2024-05-09T08:56:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.304+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8011c20 con 0x7f1dfc0e7000 2024-05-09T08:56:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1de4001030 con 0x7f1dfc0e7000 2024-05-09T08:56:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f97215da640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 0x7f97140e14a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c7a0 tx=0x7f970c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c84144db0 con 0x7f7c6c004160 2024-05-09T08:56:09.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f5d96d9d640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d780741b0 0x7f5d780765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68980cc740 con 0x7f6898156960 2024-05-09T08:56:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.305+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68980ccd00 con 0x7f6898156960 2024-05-09T08:56:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c018070 con 0x7f9714031460 2024-05-09T08:56:09.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f6edaed4640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6eb4001030 con 0x7f6ed41538d0 2024-05-09T08:56:09.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c007500 con 0x7f9714031460 2024-05-09T08:56:09.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df8018020 con 0x7f1dfc0e7000 2024-05-09T08:56:09.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ecc042440 con 0x7f6ed41538d0 2024-05-09T08:56:09.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.306+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c004a20 con 0x7f9714031460 2024-05-09T08:56:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f689804f5f0 con 0x7f6898156960 2024-05-09T08:56:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f6ed17fa640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6ec006bf30 0x7f6ec006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1deffff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1de806bf00 0x7f1de806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7fccbd77a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 0x7fccb8157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d80032440 con 0x7f5d780741b0 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1df808ae90 con 0x7f1dfc0e7000 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1deffff640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1de8074180 0x7f1de80765a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1df800fe50 con 0x7f1de8074180 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1df808b280 con 0x7f1dfc0e7000 2024-05-09T08:56:09.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f1e037fe640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1de806bf00 0x7f1de806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6ecc0ac060 con 0x7f6ed41538d0 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f6ed8c70640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6ec006bf30 0x7f6ec006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c7c00ac30 con 0x7f7c6c004160 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7fccbd77a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 0x7fccb8157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.307+0000 7fccb7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 0x7fccb8157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48098/0 (socket says 172.21.15.49:48098) 2024-05-09T08:56:09.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccb7fff640 1 -- 172.21.15.49:0/1740767357 learned_addr learned my addr 172.21.15.49:0/1740767357 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f5d8dd9b640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d7806bf30 0x7f5d7806e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d980ce280 tx=0x7f5d88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccb77fe640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f1e02ffd640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1de8074180 0x7f1de80765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccbd77a640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb81556f0 0x7fccb8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f6ed17fa640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6ec0074130 0x7f6ec0076550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f7c80ff9640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c7006bf30 0x7f7c7006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccbd77a640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fccb80cd490 con 0x7fccb80cc000 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccbd77a640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccb80ccc90 con 0x7fccb80cc450 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccbd77a640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fccb80cd610 con 0x7fccb81556f0 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb81556f0 0x7fccb8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f1e037fe640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1de806bf00 0x7f1de806e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e0402d750 tx=0x7f1e0406ae90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f1e02ffd640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1de8074180 0x7f1de80765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c7c08ad10 con 0x7f7c6c004160 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.308+0000 7f7c837fe640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c7006bf30 0x7f7c7006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ecc0020b0 con 0x7f6ec0074130 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6ecc0763c0 con 0x7f6ed41538d0 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f6ed8c70640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6ec006bf30 0x7f6ec006e3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4050860 tx=0x7f6ec4007950 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f6ed3fff640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6ec0074130 0x7f6ec0076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 -- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb81556f0 msgr2=0x7fccb8155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f9722ddd640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97140e02a0 con 0x7f9714031460 2024-05-09T08:56:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f9722ddd640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97140e0ca0 con 0x7f9714031460 2024-05-09T08:56:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb81556f0 0x7fccb8155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68900124f0 con 0x7f6898156960 2024-05-09T08:56:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 -- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 msgr2=0x7fccb815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb814e480 con 0x7fccb80cc000 2024-05-09T08:56:09.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f6ed3fff640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6ec0074130 0x7f6ec0076550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f7c80ff9640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c700741b0 0x7f7c700765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df800fe50 con 0x7f1de8074180 2024-05-09T08:56:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c7c016e50 con 0x7f7c700741b0 2024-05-09T08:56:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c08d370 con 0x7f7c6c004160 2024-05-09T08:56:09.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c700741b0 0x7f7c700765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.309+0000 7fccb7fff640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 0x7fccb8157510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c430 tx=0x7fcca800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f688e7fc640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687c06be60 0x7f687c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7fccb77fe640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccb80cc450 0x7fccb815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:09.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccb81556f0 0x7fccb8155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8017070 con 0x7fccb80cc000 2024-05-09T08:56:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f689008b2a0 con 0x7f6898156960 2024-05-09T08:56:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca80035f0 con 0x7fccb80cc000 2024-05-09T08:56:09.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f689d945640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687c06be60 0x7f687c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7fccbd77a640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb814e740 con 0x7fccb80cc000 2024-05-09T08:56:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f688e7fc640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f687c0740e0 0x7f687c076500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6890016e50 con 0x7f687c0740e0 2024-05-09T08:56:09.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f689008b690 con 0x7f6898156960 2024-05-09T08:56:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ecc0020b0 con 0x7f6ec0074130 2024-05-09T08:56:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.311+0000 7f9722ddd640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9714005380 con 0x7f9714031460 2024-05-09T08:56:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.311+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca80039e0 con 0x7fccb80cc000 2024-05-09T08:56:09.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.311+0000 7fccbd77a640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb814ed00 con 0x7fccb80cc000 2024-05-09T08:56:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.311+0000 7f689d945640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687c06be60 0x7f687c06e320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6894009ae0 tx=0x7f6894008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.311+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f970c004bc0 con 0x7f9714031460 2024-05-09T08:56:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.312+0000 7f97127fc640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f970006be60 0x7f970006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.312+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f970c08a4a0 con 0x7f9714031460 2024-05-09T08:56:09.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.312+0000 7f9721ddb640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f970006be60 0x7f970006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.310+0000 7f7c837fe640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c7006bf30 0x7f7c7006e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c84145810 tx=0x7f7c74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.312+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fccb81560e0 con 0x7fccb80cc000 2024-05-09T08:56:09.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.312+0000 7f7c88bd5640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c700741b0 0x7f7c700765d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f97127fc640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f9700074080 0x7f97000764a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9700076b50 con 0x7f9700074080 2024-05-09T08:56:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f970c091050 con 0x7f9714031460 2024-05-09T08:56:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f689d144640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f687c0740e0 0x7f687c076500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f9721ddb640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f970006be60 0x7f970006e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f971404a7b0 tx=0x7f9718008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcca801a070 con 0x7fccb80cc000 2024-05-09T08:56:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f97225dc640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f9700074080 0x7f97000764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c7c016e50 con 0x7f7c700741b0 2024-05-09T08:56:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.313+0000 7f689d144640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f687c0740e0 0x7f687c076500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb4ff9640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcca406bf30 0x7fcca406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7f97225dc640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f9700074080 0x7f97000764a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcca808a330 con 0x7fccb80cc000 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb77fe640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcca406bf30 0x7fcca406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb4ff9640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcca40741b0 0x7fcca40765d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcca4076c80 con 0x7fcca40741b0 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcca8091050 con 0x7fccb80cc000 2024-05-09T08:56:09.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6890016e50 con 0x7f687c0740e0 2024-05-09T08:56:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.314+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcca40741b0 0x7fcca40765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.315+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9700076b50 con 0x7f9700074080 2024-05-09T08:56:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.315+0000 7fccb77fe640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcca406bf30 0x7fcca406e3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccb806c660 tx=0x7fccac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.315+0000 7fccbcf79640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcca40741b0 0x7fcca40765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.316+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcca4076c80 con 0x7fcca40741b0 2024-05-09T08:56:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.337+0000 7f32fe5c8640 1 -- 172.21.15.49:0/2308747225 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32d8002dd0 con 0x7f32dc0740b0 2024-05-09T08:56:09.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.338+0000 7f32f4ff9640 1 -- 172.21.15.49:0/2308747225 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32d8002dd0 con 0x7f32dc0740b0 2024-05-09T08:56:09.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.338+0000 7f5d9759e640 1 -- 172.21.15.49:0/4056398133 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d7c002dd0 con 0x7f5d780741b0 2024-05-09T08:56:09.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f32dc0740b0 msgr2=0x7f32dc0764d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f32dc0740b0 0x7f32dc0764d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f5d8f7fe640 1 -- 172.21.15.49:0/4056398133 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5d7c002dd0 con 0x7f5d780741b0 2024-05-09T08:56:09.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f32dc06beb0 msgr2=0x7f32dc06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f32dc06beb0 0x7f32dc06e370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32e4034000 tx=0x7f32e402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:09.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 msgr2=0x7f32f0178bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32e27fc640 1 --2- 172.21.15.49:0/2308747225 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32f0050c90 0x7f32f0178bb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00eb40 tx=0x7f32ec00c150 comp rx=0 tx=0).stop 2024-05-09T08:56:09.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.339+0000 7f32fcb65640 1 -- 172.21.15.49:0/2308747225 reap_dead start 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d780741b0 msgr2=0x7f5d780765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d780741b0 0x7f5d780765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d7806bf30 msgr2=0x7f5d7806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d7806bf30 0x7f5d7806e3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d980ce280 tx=0x7f5d88008040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 msgr2=0x7f5d981f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 --2- 172.21.15.49:0/4056398133 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d9806d480 0x7f5d981f4c40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d800097f0 tx=0x7f5d800046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d96d9d640 1 -- 172.21.15.49:0/4056398133 reap_dead start 2024-05-09T08:56:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 shutdown_connections 2024-05-09T08:56:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 shutdown_connections 2024-05-09T08:56:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 >> 172.21.15.49:0/2308747225 conn(0x7f32f0050170 msgr2=0x7f32f004dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 >> 172.21.15.49:0/4056398133 conn(0x7f5d9806b200 msgr2=0x7f5d9815cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 shutdown_connections 2024-05-09T08:56:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 shutdown_connections 2024-05-09T08:56:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f32e27fc640 1 -- 172.21.15.49:0/2308747225 wait complete. 2024-05-09T08:56:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.340+0000 7f5d8cd99640 1 -- 172.21.15.49:0/4056398133 wait complete. 2024-05-09T08:56:09.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.341+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1de4002dd0 con 0x7f1de8074180 2024-05-09T08:56:09.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.341+0000 7f1deffff640 1 -- 172.21.15.49:0/1670043785 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1de4002dd0 con 0x7f1de8074180 2024-05-09T08:56:09.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1de8074180 msgr2=0x7f1de80765a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1de8074180 0x7f1de80765a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1de806bf00 msgr2=0x7f1de806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1de806bf00 0x7f1de806e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e0402d750 tx=0x7f1e0406ae90 comp rx=0 tx=0).stop 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc0e7000 msgr2=0x7f1dfc0e4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1dedffb640 1 --2- 172.21.15.49:0/1670043785 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dfc0e7000 0x7f1dfc0e4940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b410 tx=0x7f1df800b8d0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.342+0000 7f1e03fff640 1 -- 172.21.15.49:0/1670043785 reap_dead start 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.343+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 shutdown_connections 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.343+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 >> 172.21.15.49:0/1670043785 conn(0x7f1dfc04dd20 msgr2=0x7f1dfc0dc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.344+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 shutdown_connections 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.344+0000 7f1dedffb640 1 -- 172.21.15.49:0/1670043785 wait complete. 2024-05-09T08:56:09.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.345+0000 7f6edaed4640 1 -- 172.21.15.49:0/4114375728 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6eb4002dd0 con 0x7f6ec0074130 2024-05-09T08:56:09.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f6ed17fa640 1 -- 172.21.15.49:0/4114375728 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6eb4002dd0 con 0x7f6ec0074130 2024-05-09T08:56:09.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f689804f5f0 con 0x7f687c0740e0 2024-05-09T08:56:09.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c840c9570 con 0x7f7c700741b0 2024-05-09T08:56:09.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f7c80ff9640 1 -- 172.21.15.49:0/4009354850 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7c840c9570 con 0x7f7c700741b0 2024-05-09T08:56:09.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f688e7fc640 1 -- 172.21.15.49:0/2569067692 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f689804f5f0 con 0x7f687c0740e0 2024-05-09T08:56:09.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6ec0074130 msgr2=0x7f6ec0076550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.346+0000 7f6ebaffd640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6ec0074130 0x7f6ec0076550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6ec006bf30 msgr2=0x7f6ec006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6ec006bf30 0x7f6ec006e3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4050860 tx=0x7f6ec4007950 comp rx=0 tx=0).stop 2024-05-09T08:56:09.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 msgr2=0x7f6ed41efae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 --2- 172.21.15.49:0/4114375728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ed41538d0 0x7f6ed41efae0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc039d40 tx=0x7f6ecc004ab0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ed9471640 1 -- 172.21.15.49:0/4114375728 reap_dead start 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 shutdown_connections 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 >> 172.21.15.49:0/4114375728 conn(0x7f6ed4074a20 msgr2=0x7f6ed40cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f687c0740e0 msgr2=0x7f687c076500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f689fba9640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f687c0740e0 0x7f687c076500 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c700741b0 msgr2=0x7f7c700765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f7c8a638640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c700741b0 0x7f7c700765d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687c06be60 msgr2=0x7f687c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f689fba9640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687c06be60 0x7f687c06e320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6894009ae0 tx=0x7f6894008040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 msgr2=0x7f6898149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689fba9640 1 --2- 172.21.15.49:0/2569067692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6898156960 0x7f6898149ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6898072bd0 tx=0x7f689000f5a0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689e146640 1 -- 172.21.15.49:0/2569067692 reap_dead start 2024-05-09T08:56:09.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 shutdown_connections 2024-05-09T08:56:09.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 >> 172.21.15.49:0/2569067692 conn(0x7f689806ef50 msgr2=0x7f68980c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 shutdown_connections 2024-05-09T08:56:09.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.347+0000 7f6ebaffd640 1 -- 172.21.15.49:0/4114375728 wait complete. 2024-05-09T08:56:09.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f9722ddd640 1 -- 172.21.15.49:0/1359626007 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9714045420 con 0x7f9700074080 2024-05-09T08:56:09.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c7006bf30 msgr2=0x7f7c7006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c7006bf30 0x7f7c7006e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c84145810 tx=0x7f7c74008040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c004160 msgr2=0x7f7c84157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 --2- 172.21.15.49:0/4009354850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c6c004160 0x7f7c84157510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00c8d0 tx=0x7f7c7c00cd90 comp rx=0 tx=0).stop 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c88bd5640 1 -- 172.21.15.49:0/4009354850 reap_dead start 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 shutdown_connections 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f689fba9640 1 -- 172.21.15.49:0/2569067692 wait complete. 2024-05-09T08:56:09.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 shutdown_connections 2024-05-09T08:56:09.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 >> 172.21.15.49:0/4009354850 conn(0x7f7c8406b200 msgr2=0x7f7c841548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f97127fc640 1 -- 172.21.15.49:0/1359626007 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9714045420 con 0x7f9700074080 2024-05-09T08:56:09.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.348+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 shutdown_connections 2024-05-09T08:56:09.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f7c8a638640 1 -- 172.21.15.49:0/4009354850 wait complete. 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f9700074080 msgr2=0x7f97000764a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f9700074080 0x7f97000764a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f970006be60 msgr2=0x7f970006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f970006be60 0x7f970006e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f971404a7b0 tx=0x7f9718008040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 msgr2=0x7f97140e14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.349+0000 7f96fffff640 1 --2- 172.21.15.49:0/1359626007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9714031460 0x7f97140e14a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c7a0 tx=0x7f970c00cc60 comp rx=0 tx=0).stop 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7f97225dc640 1 -- 172.21.15.49:0/1359626007 reap_dead start 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 shutdown_connections 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 >> 172.21.15.49:0/1359626007 conn(0x7f9714050170 msgr2=0x7f97140d6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 shutdown_connections 2024-05-09T08:56:09.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7f96fffff640 1 -- 172.21.15.49:0/1359626007 wait complete. 2024-05-09T08:56:09.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.350+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fccb804f5f0 con 0x7fcca40741b0 2024-05-09T08:56:09.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.351+0000 7fccb4ff9640 1 -- 172.21.15.49:0/1740767357 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fccb804f5f0 con 0x7fcca40741b0 2024-05-09T08:56:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.355+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcca40741b0 msgr2=0x7fcca40765d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.355+0000 7fcc9e7fc640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcca40741b0 0x7fcca40765d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.356+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcca406bf30 msgr2=0x7fcca406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.356+0000 7fcc9e7fc640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcca406bf30 0x7fcca406e3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccb806c660 tx=0x7fccac008040 comp rx=0 tx=0).stop 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.356+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 msgr2=0x7fccb8157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.356+0000 7fcc9e7fc640 1 --2- 172.21.15.49:0/1740767357 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fccb80cc000 0x7fccb8157510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c430 tx=0x7fcca800c8f0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.356+0000 7fccbcf79640 1 -- 172.21.15.49:0/1740767357 reap_dead start 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.358+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 shutdown_connections 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.358+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 >> 172.21.15.49:0/1740767357 conn(0x7fccb8067760 msgr2=0x7fccb8072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.359+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 shutdown_connections 2024-05-09T08:56:09.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.359+0000 7fcc9e7fc640 1 -- 172.21.15.49:0/1740767357 wait complete. 2024-05-09T08:56:09.417 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:09.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:56:09.418 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-05-09T08:56:09.418 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:56:09.419 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:09.419 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:56:09.420 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:09.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:56:09.421 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:09.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:56:09.422 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-05-09T08:56:09.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:56:09.423 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:09.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:56:09.424 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:09.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:56:09.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.636+0000 7fdbde4a5640 1 Processor -- start 2024-05-09T08:56:09.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.637+0000 7fdbde4a5640 1 -- start start 2024-05-09T08:56:09.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7f4b4ee33640 1 Processor -- start 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbde4a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 0x7fdbd8070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbde4a5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdbd80c6300 0x7fdbd8169890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbde4a5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbd80c7820 con 0x7fdbd8070580 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbde4a5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdbd80c9c20 con 0x7fdbd80c6300 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbde4a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd80c70b0 con 0x7fdbd81593e0 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7f4b4ee33640 1 -- start start 2024-05-09T08:56:09.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbdca42640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdbd80c6300 0x7fdbd8169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbdca42640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdbd80c6300 0x7fdbd8169890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55446/0 (socket says 172.21.15.49:55446) 2024-05-09T08:56:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbdca42640 1 -- 172.21.15.49:0/3531905169 learned_addr learned my addr 172.21.15.49:0/3531905169 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7fdbd7fff640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 0x7fdbd8070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.638+0000 7f3ed1d14640 1 Processor -- start 2024-05-09T08:56:09.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f3ed1d14640 1 -- start start 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f4b4ee33640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4b400dd4b0 0x7f4b40007c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f4b4ee33640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4b400471a0 0x7f4b40047630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f4b4ee33640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b40047fa0 con 0x7f4b4004bc90 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f4b4ee33640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4b40006e70 con 0x7f4b400dd4b0 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7f4b4ee33640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4b40007150 con 0x7f4b400471a0 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f4b4d630640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4b4004bc90 0x7f4b40046440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36402/0 (socket says 172.21.15.49:36402) 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f4b4d630640 1 -- 172.21.15.49:0/293420694 learned_addr learned my addr 172.21.15.49:0/293420694 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7fdbdca42640 1 -- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 msgr2=0x7fdbd8070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7fdbdca42640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 0x7fdbd8070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7fdbdca42640 1 -- 172.21.15.49:0/3531905169 >> v1:172.21.15.183:6789/0 conn(0x7fdbd8070580 legacy=0x7fdbd8162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.639+0000 7fdbdca42640 1 -- 172.21.15.49:0/3531905169 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd8163410 con 0x7fdbd80c6300 2024-05-09T08:56:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f8821599640 1 Processor -- start 2024-05-09T08:56:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f4b4e632640 1 --2- 172.21.15.49:0/293420694 >> v2:172.21.15.183:3300/0 conn(0x7f4b400471a0 0x7f4b40047630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f8821599640 1 -- start start 2024-05-09T08:56:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> v2:172.21.15.49:3300/0 conn(0x7f4b400dd4b0 0x7f4b40007c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ed1d14640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ec404b090 0x7f3ec404b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ed1d14640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3ec40d4f90 0x7f3ec4005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ecaffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ec404b090 0x7f3ec404b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ecaffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3ec404b090 0x7f3ec404b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48108/0 (socket says 172.21.15.49:48108) 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3887558723 learned_addr learned my addr 172.21.15.49:0/3887558723 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec4046f30 con 0x7f3ec400e2a0 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec4046170 con 0x7f3ec404b090 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec4046450 con 0x7f3ec40d4f90 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812501246 0 0) 0x7f4b40047fa0 con 0x7f4b4004bc90 2024-05-09T08:56:09.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7fdbd7fff640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 0x7fdbd8070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ecbfff640 1 --2- 172.21.15.49:0/3887558723 >> v2:172.21.15.183:3300/0 conn(0x7f3ec40d4f90 0x7f3ec4005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.640+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3887558723 >> v2:172.21.15.183:3300/0 conn(0x7f3ec40d4f90 msgr2=0x7f3ec4005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f3ecaffd640 1 --2- 172.21.15.49:0/3887558723 >> v2:172.21.15.183:3300/0 conn(0x7f3ec40d4f90 0x7f3ec4005e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3887558723 >> v1:172.21.15.49:6789/0 conn(0x7f3ec400e2a0 legacy=0x7f3ec40db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3887558723 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec4046730 con 0x7f3ec404b090 2024-05-09T08:56:09.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b34003520 con 0x7f4b4004bc90 2024-05-09T08:56:09.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7fdbdca42640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd80c6300 0x7fdbd8169890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e50 tx=0x7fdbd002a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10eb9e2b7646c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f6f21763640 1 Processor -- start 2024-05-09T08:56:09.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f73c155a640 1 Processor -- start 2024-05-09T08:56:09.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887318407 0 0) 0x7f4b34003520 con 0x7f4b4004bc90 2024-05-09T08:56:09.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f6f21763640 1 -- start start 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 >> v2:172.21.15.183:3300/0 conn(0x7f4b400471a0 msgr2=0x7f4b40047630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 --2- 172.21.15.49:0/293420694 >> v2:172.21.15.183:3300/0 conn(0x7f4b400471a0 0x7f4b40047630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 >> v2:172.21.15.49:3300/0 conn(0x7f4b400dd4b0 msgr2=0x7f4b40007c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 --2- 172.21.15.49:0/293420694 >> v2:172.21.15.49:3300/0 conn(0x7f4b400dd4b0 0x7f4b40007c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40007430 con 0x7f4b4004bc90 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f4b4806a1d0 con 0x7f4b4004bc90 2024-05-09T08:56:09.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> v2:172.21.15.49:3300/0 conn(0x7f4b400dd4b0 0x7f4b40007c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7fdbd6ffd640 1 -- 172.21.15.49:0/3531905169 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0036070 con 0x7fdbd80c6300 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7fdbd6ffd640 1 -- 172.21.15.49:0/3531905169 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd0031600 con 0x7fdbd80c6300 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f8821599640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8814045720 0x7f8814045b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f8821599640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f88140de3b0 0x7f88140de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f8821599640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8814046090 con 0x7f88140d5090 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f8821599640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8814007aa0 con 0x7f88140de3b0 2024-05-09T08:56:09.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f8821599640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8814006e90 con 0x7f8814045720 2024-05-09T08:56:09.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7fdbd6ffd640 1 -- 172.21.15.49:0/3531905169 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0031a10 con 0x7fdbd80c6300 2024-05-09T08:56:09.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f881bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f88140d5090 0x7f881404b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:45976/0 (socket says 172.21.15.49:45976) 2024-05-09T08:56:09.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f881bfff640 1 -- 172.21.15.49:0/1669705589 learned_addr learned my addr 172.21.15.49:0/1669705589 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f8820d98640 1 --2- 172.21.15.49:0/1669705589 >> v2:172.21.15.49:3300/0 conn(0x7f88140de3b0 0x7f88140de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 >> v1:172.21.15.183:6789/0 conn(0x7f4b4004bc90 legacy=0x7f4b40046440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.641+0000 7f73c155a640 1 -- start start 2024-05-09T08:56:09.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f3ecaffd640 1 --2- 172.21.15.49:0/3887558723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec404b620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec404c080 tx=0x7f3ec002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211c0d4e6ceb67ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4ce2f640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f881b7fe640 1 --2- 172.21.15.49:0/1669705589 >> v2:172.21.15.183:3300/0 conn(0x7f8814045720 0x7f8814045b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4ce2f640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b34004310 0x7f4b34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4d630640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b34004310 0x7f4b34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f00e1170640 1 Processor -- start 2024-05-09T08:56:09.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f73c155a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f73bc0ccdb0 0x7f73bc151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f4b4ce2f640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b34004e60 0x7f4b3400b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f6f21763640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 0x7f6f1c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f73c155a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc0c7910 0x7f73bc0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f73c155a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73bc151540 con 0x7f73bc06dac0 2024-05-09T08:56:09.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f73c155a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f73bc157d20 con 0x7f73bc0ccdb0 2024-05-09T08:56:09.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f00e1170640 1 -- start start 2024-05-09T08:56:09.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f73c155a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc1571a0 con 0x7f73bc0c7910 2024-05-09T08:56:09.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4e632640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b34004e60 0x7f4b3400b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73bb7fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f73bc06dac0 0x7f73bc15a940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36408/0 (socket says 172.21.15.49:36408) 2024-05-09T08:56:09.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73bb7fe640 1 -- 172.21.15.49:0/614451603 learned_addr learned my addr 172.21.15.49:0/614451603 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:09.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73baffd640 1 --2- 172.21.15.49:0/614451603 >> v2:172.21.15.49:3300/0 conn(0x7f73bc0ccdb0 0x7f73bc151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40006e70 con 0x7f4b34004310 2024-05-09T08:56:09.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40007150 con 0x7f4b34004e60 2024-05-09T08:56:09.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3400bef0 con 0x7f4b340039b0 2024-05-09T08:56:09.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73ba7fc640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc0c7910 0x7f73bc0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f3eca7fc640 1 -- 172.21.15.49:0/3887558723 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0037070 con 0x7f3ec404b090 2024-05-09T08:56:09.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f3eca7fc640 1 -- 172.21.15.49:0/3887558723 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec0032440 con 0x7f3ec404b090 2024-05-09T08:56:09.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.642+0000 7f3eca7fc640 1 -- 172.21.15.49:0/3887558723 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0032810 con 0x7f3ec404b090 2024-05-09T08:56:09.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73baffd640 1 -- 172.21.15.49:0/614451603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc0c7910 msgr2=0x7f73bc0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f881b7fe640 1 -- 172.21.15.49:0/1669705589 >> v2:172.21.15.49:3300/0 conn(0x7f88140de3b0 msgr2=0x7f88140de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f881b7fe640 1 --2- 172.21.15.49:0/1669705589 >> v2:172.21.15.49:3300/0 conn(0x7f88140de3b0 0x7f88140de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f881b7fe640 1 -- 172.21.15.49:0/1669705589 >> v1:172.21.15.49:6789/0 conn(0x7f88140d5090 legacy=0x7f881404b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f881b7fe640 1 -- 172.21.15.49:0/1669705589 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8814007170 con 0x7f8814045720 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73baffd640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc0c7910 0x7f73bc0c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73b9ffb640 1 -- 172.21.15.49:0/614451603 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278336895 0 0) 0x7f73bc151540 con 0x7f73bc06dac0 2024-05-09T08:56:09.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f73baffd640 1 -- 172.21.15.49:0/614451603 >> v1:172.21.15.183:6789/0 conn(0x7f73bc06dac0 legacy=0x7f73bc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4de31640 1 -- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b34004e60 msgr2=0x7f4b3400b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b34004e60 0x7f4b3400b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4de31640 1 -- 172.21.15.49:0/293420694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b34004310 msgr2=0x7f4b34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b34004310 0x7f4b34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f4b4de31640 1 -- 172.21.15.49:0/293420694 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b34003520 con 0x7f4b340039b0 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7fdbde4a5640 1 -- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd80c6300 msgr2=0x7fdbd8169890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd80c6300 0x7fdbd8169890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009e50 tx=0x7fdbd002a730 comp rx=0 tx=0).stop 2024-05-09T08:56:09.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f73baffd640 1 -- 172.21.15.49:0/614451603 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc157480 con 0x7f73bc0ccdb0 2024-05-09T08:56:09.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 -- 172.21.15.49:0/3531905169 shutdown_connections 2024-05-09T08:56:09.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd80c6300 0x7fdbd8169890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/3531905169 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd81593e0 0x7fdbd8070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 -- 172.21.15.49:0/3531905169 >> 172.21.15.49:0/3531905169 conn(0x7fdbd8074840 msgr2=0x7fdbd8074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f21763640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6f1c0c7130 0x7f6f1c0c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f21763640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c0c6450 0x7f6f1c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f21763640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c159b10 con 0x7f6f1c0c7130 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f21763640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1c159270 con 0x7f6f1c152e30 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 msgr2=0x7f3ec404b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.643+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3887558723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec404b620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ec404c080 tx=0x7f3ec002a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 shutdown_connections 2024-05-09T08:56:09.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f21763640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c159550 con 0x7f6f1c0c6450 2024-05-09T08:56:09.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 -- 172.21.15.49:0/3531905169 shutdown_connections 2024-05-09T08:56:09.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3887558723 >> v2:172.21.15.183:3300/0 conn(0x7f3ec40d4f90 0x7f3ec4005e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3887558723 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec404b620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 >> 172.21.15.49:0/3887558723 conn(0x7f3ec4050180 msgr2=0x7f3ec4050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 shutdown_connections 2024-05-09T08:56:09.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f00e1170640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 0x7f00dc0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f00e1170640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 0x7f00dc06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f00e1170640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00dc157d20 con 0x7f00dc06e000 2024-05-09T08:56:09.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f00e1170640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f00dc1571a0 con 0x7f00dc0c7910 2024-05-09T08:56:09.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f00e1170640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc157480 con 0x7f00dc151000 2024-05-09T08:56:09.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f4b4e632640 1 -- 172.21.15.49:0/293420694 reap_dead start 2024-05-09T08:56:09.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7fdbde4a5640 1 -- 172.21.15.49:0/3531905169 wait complete. 2024-05-09T08:56:09.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f881b7fe640 1 --2- 172.21.15.49:0/1669705589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f8814045b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8814051440 tx=0x7f880802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a65e21984619ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f00da59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 0x7f00dc06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f4b4de31640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b34003d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b34005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d5213db7a07568 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f00da59c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 0x7f00dc06dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55502/0 (socket says 172.21.15.49:55502) 2024-05-09T08:56:09.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f00da59c640 1 -- 172.21.15.49:0/1200437075 learned_addr learned my addr 172.21.15.49:0/1200437075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400f040 con 0x7f4b340039b0 2024-05-09T08:56:09.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f00dad9d640 1 --2- 172.21.15.49:0/1200437075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 0x7f00dc0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f1affd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 0x7f6f1c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f1affd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 0x7f6f1c158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55494/0 (socket says 172.21.15.49:55494) 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f1affd640 1 -- 172.21.15.49:0/210541990 learned_addr learned my addr 172.21.15.49:0/210541990 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f1b7fe640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c0c6450 0x7f6f1c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f6f1a7fc640 1 --2- 172.21.15.49:0/210541990 >> v2:172.21.15.49:3300/0 conn(0x7f6f1c0c7130 0x7f6f1c0c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f6f1a7fc640 1 -- 172.21.15.49:0/210541990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c0c6450 msgr2=0x7f6f1c0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f6f1a7fc640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c0c6450 0x7f6f1c0c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f6f1a7fc640 1 -- 172.21.15.49:0/210541990 >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 msgr2=0x7f6f1c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f6f1a7fc640 1 --2- 172.21.15.49:0/210541990 >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 0x7f6f1c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f6f1a7fc640 1 -- 172.21.15.49:0/210541990 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c159830 con 0x7f6f1c0c7130 2024-05-09T08:56:09.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f4b4ce2f640 1 -- 172.21.15.49:0/293420694 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4402acb0 con 0x7f4b340039b0 2024-05-09T08:56:09.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f881affd640 1 -- 172.21.15.49:0/1669705589 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808037070 con 0x7f8814045720 2024-05-09T08:56:09.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f881affd640 1 -- 172.21.15.49:0/1669705589 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808032440 con 0x7f8814045720 2024-05-09T08:56:09.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.645+0000 7f73baffd640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc151000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009a00 tx=0x7f73ac02a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00bad0d5cf1b6c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7fdbde4a5640 1 Processor -- start 2024-05-09T08:56:09.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f73b9ffb640 1 -- 172.21.15.49:0/614451603 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac036070 con 0x7f73bc0ccdb0 2024-05-09T08:56:09.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f73b9ffb640 1 -- 172.21.15.49:0/614451603 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac031600 con 0x7f73bc0ccdb0 2024-05-09T08:56:09.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7fdbde4a5640 1 -- start start 2024-05-09T08:56:09.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f6f1a7fc640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c0c5f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0cce70 tx=0x7f6f0402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6d262eb01cba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.644+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3887558723 wait complete. 2024-05-09T08:56:09.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f3ed1d14640 1 Processor -- start 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 -- 172.21.15.49:0/1669705589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 msgr2=0x7f8814045b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 --2- 172.21.15.49:0/1669705589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f8814045b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8814051440 tx=0x7f880802a230 comp rx=0 tx=0).stop 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f4b4ee33640 1 -- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 msgr2=0x7f4b34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f4b4ee33640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b34003d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098a0 tx=0x7f4b34005830 comp rx=0 tx=0).stop 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f4b4ee33640 1 -- 172.21.15.49:0/293420694 shutdown_connections 2024-05-09T08:56:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f4b4ee33640 1 --2- 172.21.15.49:0/293420694 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b34003d90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f4b4ee33640 1 -- 172.21.15.49:0/293420694 >> 172.21.15.49:0/293420694 conn(0x7f4b40051310 msgr2=0x7f4b40051720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f6f19ffb640 1 -- 172.21.15.49:0/210541990 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04037070 con 0x7f6f1c0c7130 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f00dad9d640 1 -- 172.21.15.49:0/1200437075 >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 msgr2=0x7f00dc06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f00dad9d640 1 --2- 172.21.15.49:0/1200437075 >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 0x7f00dc06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f00dad9d640 1 -- 172.21.15.49:0/1200437075 >> v1:172.21.15.49:6789/0 conn(0x7f00dc06e000 legacy=0x7f00dc15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f00dad9d640 1 -- 172.21.15.49:0/1200437075 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc157760 con 0x7f00dc151000 2024-05-09T08:56:09.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 -- 172.21.15.49:0/1669705589 shutdown_connections 2024-05-09T08:56:09.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 --2- 172.21.15.49:0/1669705589 >> v2:172.21.15.49:3300/0 conn(0x7f88140de3b0 0x7f88140de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 --2- 172.21.15.49:0/1669705589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f8814045b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f8821599640 1 -- 172.21.15.49:0/1669705589 >> 172.21.15.49:0/1669705589 conn(0x7f88140505d0 msgr2=0x7f88140509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f6f19ffb640 1 -- 172.21.15.49:0/210541990 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f04032440 con 0x7f6f1c0c7130 2024-05-09T08:56:09.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f4b4ee33640 1 -- 172.21.15.49:0/293420694 shutdown_connections 2024-05-09T08:56:09.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f6f19ffb640 1 -- 172.21.15.49:0/210541990 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04032810 con 0x7f6f1c0c7130 2024-05-09T08:56:09.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f560cdff640 1 Processor -- start 2024-05-09T08:56:09.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f8821599640 1 -- 172.21.15.49:0/1669705589 shutdown_connections 2024-05-09T08:56:09.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f560cdff640 1 -- start start 2024-05-09T08:56:09.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 -- 172.21.15.49:0/614451603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 msgr2=0x7f73bc151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc151000 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009a00 tx=0x7f73ac02a490 comp rx=0 tx=0).stop 2024-05-09T08:56:09.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f00dad9d640 1 --2- 172.21.15.49:0/1200437075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 0x7f00dc0c6c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009d90 tx=0x7f00d402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0aff61fbe0ccff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.646+0000 7f3ed1d14640 1 -- start start 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd8070580 0x7fdbd80c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd80c6300 0x7fdbd81616d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd815fa30 con 0x7fdbd81593e0 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd80c7390 con 0x7fdbd8070580 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7fdbde4a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdbd815fd10 con 0x7fdbd80c6300 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 -- 172.21.15.49:0/614451603 shutdown_connections 2024-05-09T08:56:09.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc0c7910 0x7f73bc0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 --2- 172.21.15.49:0/614451603 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc151000 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 -- 172.21.15.49:0/614451603 >> 172.21.15.49:0/614451603 conn(0x7f73bc0cc290 msgr2=0x7f73bc0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f00d9d9b640 1 -- 172.21.15.49:0/1200437075 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4007740 con 0x7f00dc151000 2024-05-09T08:56:09.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f00d9d9b640 1 -- 172.21.15.49:0/1200437075 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d40078e0 con 0x7f00dc151000 2024-05-09T08:56:09.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 -- 172.21.15.49:0/614451603 shutdown_connections 2024-05-09T08:56:09.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f00d9d9b640 1 -- 172.21.15.49:0/1200437075 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4007c90 con 0x7f00dc151000 2024-05-09T08:56:09.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f4b4ee33640 1 -- 172.21.15.49:0/293420694 wait complete. 2024-05-09T08:56:09.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48156/0 (socket says 172.21.15.49:48156) 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 -- 172.21.15.49:0/4118298801 learned_addr learned my addr 172.21.15.49:0/4118298801 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 -- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd80c6300 msgr2=0x7fdbd81616d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd80c6300 0x7fdbd81616d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 -- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd8070580 msgr2=0x7fdbd80c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd8070580 0x7fdbd80c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7fdbdca42640 1 -- 172.21.15.49:0/4118298801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd80c0200 con 0x7fdbd81593e0 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7fdbdca42640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009f70 tx=0x7fdbd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 0x7f3ec40497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48164/0 (socket says 172.21.15.49:48164) 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 0x7f3ec40497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3053948619 learned_addr learned my addr 172.21.15.49:0/3053948619 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec40d4f90 0x7f3ec40c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40d5560 con 0x7f3ec404b090 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40d53e0 con 0x7f3ec400e2a0 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec40063d0 con 0x7f3ec40d4f90 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f00e1170640 1 -- 172.21.15.49:0/1200437075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 msgr2=0x7f00dc0c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f00e1170640 1 --2- 172.21.15.49:0/1200437075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 0x7f00dc0c6c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009d90 tx=0x7f00d402a6c0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f8821599640 1 -- 172.21.15.49:0/1669705589 wait complete. 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f8821599640 1 Processor -- start 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f560cdff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806dce0 0x7f56080c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f560cdff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f560806e0c0 0x7f5608150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f560cdff640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56081570f0 con 0x7f56080c7030 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f560cdff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5608157c50 con 0x7f560806e0c0 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f560806e0c0 0x7f5608150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f560806e0c0 0x7f5608150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48170/0 (socket says 172.21.15.49:48170) 2024-05-09T08:56:09.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 -- 172.21.15.49:0/3763555772 learned_addr learned my addr 172.21.15.49:0/3763555772 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f56081573b0 con 0x7f560806dce0 2024-05-09T08:56:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f56077fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806dce0 0x7f56080c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0041020 con 0x7fdbd81593e0 2024-05-09T08:56:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd80c04c0 con 0x7fdbd81593e0 2024-05-09T08:56:09.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecbfff640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec40d4f90 0x7f3ec40c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 -- 172.21.15.49:0/3763555772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806dce0 msgr2=0x7f56080c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 --2- 172.21.15.49:0/3763555772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806dce0 0x7f56080c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 -- 172.21.15.49:0/3763555772 >> v1:172.21.15.183:6789/0 conn(0x7f56080c7030 legacy=0x7f560815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f5606ffd640 1 -- 172.21.15.49:0/3763555772 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5608157690 con 0x7f560806e0c0 2024-05-09T08:56:09.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f8821599640 1 -- start start 2024-05-09T08:56:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd80c0a80 con 0x7fdbd81593e0 2024-05-09T08:56:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.647+0000 7f73c155a640 1 -- 172.21.15.49:0/614451603 wait complete. 2024-05-09T08:56:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f73c155a640 1 Processor -- start 2024-05-09T08:56:09.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec40d4f90 msgr2=0x7f3ec40c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f73c155a640 1 -- start start 2024-05-09T08:56:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec40d4f90 0x7f3ec40c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 -- 172.21.15.49:0/210541990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 msgr2=0x7f6f1c0c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c0c5f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0cce70 tx=0x7f6f0402a140 comp rx=0 tx=0).stop 2024-05-09T08:56:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 msgr2=0x7f3ec40497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.648+0000 7f3ecaffd640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 0x7f3ec40497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 -- 172.21.15.49:0/210541990 shutdown_connections 2024-05-09T08:56:09.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c0c6450 0x7f6f1c0c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 --2- 172.21.15.49:0/210541990 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c0c5f10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 --2- 172.21.15.49:0/210541990 >> v2:172.21.15.183:3300/0 conn(0x7f6f1c152e30 0x7f6f1c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f6f21763640 1 -- 172.21.15.49:0/210541990 >> 172.21.15.49:0/210541990 conn(0x7f6f1c0cc290 msgr2=0x7f6f1c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f3ecaffd640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec40cb8e0 con 0x7f3ec404b090 2024-05-09T08:56:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f3ecaffd640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ec002a8b0 tx=0x7f3ec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f3ecb7fe640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 0x7f3ec40497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0037040 con 0x7f3ec404b090 2024-05-09T08:56:09.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f88140ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ec0042070 con 0x7f3ec404b090 2024-05-09T08:56:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 0x7f881417a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec003d440 con 0x7f3ec404b090 2024-05-09T08:56:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec40ceed0 con 0x7f3ec404b090 2024-05-09T08:56:09.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f00e1170640 1 -- 172.21.15.49:0/1200437075 shutdown_connections 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f00e1170640 1 --2- 172.21.15.49:0/1200437075 >> v2:172.21.15.183:3300/0 conn(0x7f00dc0c7910 0x7f00dc06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f00e1170640 1 --2- 172.21.15.49:0/1200437075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc151000 0x7f00dc0c6c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f00e1170640 1 -- 172.21.15.49:0/1200437075 >> 172.21.15.49:0/1200437075 conn(0x7f00dc0cc290 msgr2=0x7f00dc0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88140de3b0 0x7f88140d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f88140d5470 con 0x7f88140d5090 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8814007730 con 0x7f8814045720 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f8821599640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f881404cb50 con 0x7f88140de3b0 2024-05-09T08:56:09.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f4b4ee33640 1 Processor -- start 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.649+0000 7f4b4ee33640 1 -- start start 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7f5606ffd640 1 --2- 172.21.15.49:0/3763555772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560806e0c0 0x7f5608150a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55f800aa10 tx=0x7f55f802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898cd5cfd421551d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 0x7f73bc13ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 0x7f73bc159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc15a420 con 0x7f73bc0ccdb0 2024-05-09T08:56:09.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc157a40 con 0x7f73bc0c7910 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73c155a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f73bc0c71d0 con 0x7f73bc06dac0 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73ba7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 0x7f73bc159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73ba7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 0x7f73bc159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55532/0 (socket says 172.21.15.49:55532) 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f73ba7fc640 1 -- 172.21.15.49:0/266266096 learned_addr learned my addr 172.21.15.49:0/266266096 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b40036880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 0x7f4b40034f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4d630640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 0x7f4b40034f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 0x7f4b400e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40048280 con 0x7f4b40034210 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4d630640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 0x7f4b40034f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48186/0 (socket says 172.21.15.49:48186) 2024-05-09T08:56:09.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4d630640 1 -- 172.21.15.49:0/3210334131 learned_addr learned my addr 172.21.15.49:0/3210334131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400dd890 con 0x7f4b400354d0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400e6890 con 0x7f4b340039b0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4e632640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 0x7f4b400e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbd0037040 con 0x7fdbd81593e0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.650+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0036070 con 0x7fdbd81593e0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7fdbce7fc640 1 -- 172.21.15.49:0/4118298801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd80c9820 con 0x7fdbd81593e0 2024-05-09T08:56:09.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec40cb600 con 0x7f3ec404b090 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f6f21763640 1 -- 172.21.15.49:0/210541990 shutdown_connections 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f6f21763640 1 -- 172.21.15.49:0/210541990 wait complete. 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f8820d98640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88140de3b0 0x7f88140d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f881b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 0x7f881417a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f881bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f88140ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f881b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 0x7f881417a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48184/0 (socket says 172.21.15.49:48184) 2024-05-09T08:56:09.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f881b7fe640 1 -- 172.21.15.49:0/2653289468 learned_addr learned my addr 172.21.15.49:0/2653289468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.651+0000 7f4b4de31640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b40036880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f56067fc640 1 -- 172.21.15.49:0/3763555772 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f800b040 con 0x7f560806e0c0 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f56067fc640 1 -- 172.21.15.49:0/3763555772 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f8031710 con 0x7f560806e0c0 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f56067fc640 1 -- 172.21.15.49:0/3763555772 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8031ae0 con 0x7f560806e0c0 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 -- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88140de3b0 msgr2=0x7f88140d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88140de3b0 0x7f88140d2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 -- 172.21.15.49:0/2653289468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 msgr2=0x7f881417a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 0x7f881417a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 -- 172.21.15.49:0/2653289468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88140cecc0 con 0x7f8814045720 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4de31640 1 -- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 msgr2=0x7f4b400e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4de31640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 0x7f4b400e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4de31640 1 -- 172.21.15.49:0/3210334131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 msgr2=0x7f4b40034f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4de31640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 0x7f4b40034f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4de31640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4017ca40 con 0x7f4b340039b0 2024-05-09T08:56:09.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f881bfff640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f88140ce1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f881000c850 tx=0x7f881000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f4b4e632640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 0x7f4b400e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f73baffd640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 0x7f73bc13ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b4de31640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b40036880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005770 tx=0x7f4b44004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560806e0c0 msgr2=0x7f5608150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f560cdff640 1 --2- 172.21.15.49:0/3763555772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560806e0c0 0x7f5608150a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55f800aa10 tx=0x7f55f802c7a0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f73bb7fe640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec4005a20 con 0x7f3ec404b090 2024-05-09T08:56:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.652+0000 7f00e1170640 1 -- 172.21.15.49:0/1200437075 shutdown_connections 2024-05-09T08:56:09.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810008040 con 0x7f8814045720 2024-05-09T08:56:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f00e1170640 1 -- 172.21.15.49:0/1200437075 wait complete. 2024-05-09T08:56:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44041070 con 0x7f4b340039b0 2024-05-09T08:56:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4403d430 con 0x7f4b340039b0 2024-05-09T08:56:09.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4402ae60 con 0x7f4b340039b0 2024-05-09T08:56:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f6f21763640 1 Processor -- start 2024-05-09T08:56:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4017cd00 con 0x7f4b340039b0 2024-05-09T08:56:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b40178fb0 con 0x7f4b340039b0 2024-05-09T08:56:09.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.654+0000 7f6f21763640 1 -- start start 2024-05-09T08:56:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f73baffd640 1 -- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 msgr2=0x7f73bc159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f73baffd640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 0x7f73bc159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f73baffd640 1 -- 172.21.15.49:0/266266096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 msgr2=0x7f73bc14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f73baffd640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc14e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f73baffd640 1 -- 172.21.15.49:0/266266096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc150790 con 0x7f73bc06dac0 2024-05-09T08:56:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88140d7e90 con 0x7f8814045720 2024-05-09T08:56:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.653+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88140d8450 con 0x7f8814045720 2024-05-09T08:56:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.654+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810007950 con 0x7f8814045720 2024-05-09T08:56:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.654+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbd0044020 con 0x7fdbd81593e0 2024-05-09T08:56:09.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7fdbd4ff9640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 0x7fdbb806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73baffd640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 0x7f73bc13ed40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f73ac031000 tx=0x7f73ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdbd00abfa0 con 0x7fdbd81593e0 2024-05-09T08:56:09.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ec003d5e0 con 0x7f3ec404b090 2024-05-09T08:56:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f1c0c6450 0x7f6f1c1f0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac041020 con 0x7f73bc06dac0 2024-05-09T08:56:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b400057e0 con 0x7f4b340039b0 2024-05-09T08:56:09.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc1471a0 con 0x7f73bc06dac0 2024-05-09T08:56:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc13e260 con 0x7f73bc06dac0 2024-05-09T08:56:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 0x7f6f1c1f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac037040 con 0x7f73bc06dac0 2024-05-09T08:56:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac036070 con 0x7f73bc06dac0 2024-05-09T08:56:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f3eb7fff640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 0x7f3eb006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810007d60 con 0x7f8814045720 2024-05-09T08:56:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc005290 con 0x7f8814045720 2024-05-09T08:56:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f1a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48208/0 (socket says 172.21.15.49:48208) 2024-05-09T08:56:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c06b4b0 con 0x7f6f1c0c7130 2024-05-09T08:56:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f1a7fc640 1 -- 172.21.15.49:0/1312239112 learned_addr learned my addr 172.21.15.49:0/1312239112 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c0c59f0 con 0x7f6f1c0c6450 2024-05-09T08:56:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 shutdown_connections 2024-05-09T08:56:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 --2- 172.21.15.49:0/3763555772 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560806e0c0 0x7f5608150a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 --2- 172.21.15.49:0/3763555772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806dce0 0x7f56080c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 >> 172.21.15.49:0/3763555772 conn(0x7f5608067b50 msgr2=0x7f560806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:09.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3ec00abc20 con 0x7f3ec404b090 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f00e1170640 1 Processor -- start 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f00e1170640 1 -- start start 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f3ecb7fe640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 0x7f3eb006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7fdbd7fff640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 0x7fdbb806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 shutdown_connections 2024-05-09T08:56:09.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.655+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1c153210 con 0x7f6f1c152e30 2024-05-09T08:56:09.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f560cdff640 1 -- 172.21.15.49:0/3763555772 wait complete. 2024-05-09T08:56:09.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f6f1b7fe640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 0x7f6f1c1f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.656+0000 7f6f1affd640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f1c0c6450 0x7f6f1c1f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739c005290 con 0x7f73bc06dac0 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 -- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f1c0c6450 msgr2=0x7f6f1c1f0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f1c0c6450 0x7f6f1c1f0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 -- 172.21.15.49:0/1312239112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 msgr2=0x7f6f1c148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c14f070 con 0x7f6f1c152e30 2024-05-09T08:56:09.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f881000f400 con 0x7f8814045720 2024-05-09T08:56:09.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1a7fc640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:09.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7fdbd7fff640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 0x7fdbb806e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006fd0 tx=0x7fdbc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f6f1b7fe640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 0x7f6f1c1f32c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f14007d40 tx=0x7f6f1400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f3ecb7fe640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 0x7f3eb006e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc050ef0 tx=0x7f3ecc06a510 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f00e1170640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.657+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b44044070 con 0x7f4b340039b0 2024-05-09T08:56:09.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00e1170640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc0c7910 0x7f00dc1f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00e1170640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00dc151000 0x7f00dc1359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00e1170640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0c71d0 con 0x7f00dc06e000 2024-05-09T08:56:09.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f8818ff9640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 0x7f880006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00e1170640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc15a420 con 0x7f00dc151000 2024-05-09T08:56:09.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00e1170640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc156dd0 con 0x7f00dc0c7910 2024-05-09T08:56:09.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f881008ae00 con 0x7f8814045720 2024-05-09T08:56:09.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48222/0 (socket says 172.21.15.49:48222) 2024-05-09T08:56:09.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f4b3e7fc640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 0x7f4b2c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 -- 172.21.15.49:0/636957257 learned_addr learned my addr 172.21.15.49:0/636957257 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f881b7fe640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 0x7f880006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00db59e640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00dc151000 0x7f00dc1359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4b440aba40 con 0x7f4b340039b0 2024-05-09T08:56:09.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 -- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc0c7910 msgr2=0x7f00dc1f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc0c7910 0x7f00dc1f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 -- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00dc151000 msgr2=0x7f00dc1359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00dc151000 0x7f00dc1359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 -- 172.21.15.49:0/636957257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc0ca090 con 0x7f00dc06e000 2024-05-09T08:56:09.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f560cdff640 1 Processor -- start 2024-05-09T08:56:09.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f4b4d630640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 0x7f4b2c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.658+0000 7f00dad9d640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009d90 tx=0x7f00d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 -- start start 2024-05-09T08:56:09.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f881b7fe640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 0x7f880006e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8814045c30 tx=0x7f880802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f4b4d630640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 0x7f4b2c06e3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b48065110 tx=0x7f4b48062d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 0x7f56081ed160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 0x7f5608159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 0x7f56080cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56080c8c80 con 0x7f56080c7030 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560815a360 con 0x7f560806dce0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f560cdff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f56080c8e00 con 0x7f560806e0c0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.659+0000 7f5607fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 0x7f56080cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 0x7f56081ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f5606ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 0x7f5608159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f5607fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 0x7f56080cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48236/0 (socket says 172.21.15.49:48236) 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f5607fff640 1 -- 172.21.15.49:0/2935518195 learned_addr learned my addr 172.21.15.49:0/2935518195 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 -- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 msgr2=0x7f5608159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 0x7f5608159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 -- 172.21.15.49:0/2935518195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 msgr2=0x7f56080cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 0x7f56080cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 -- 172.21.15.49:0/2935518195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56081454f0 con 0x7f560806dce0 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f5606ffd640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 0x7f5608159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.660+0000 7f56077fe640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 0x7f56081ed160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d5f0 tx=0x7f55fc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.661+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdbd80c9820 con 0x7fdbd81593e0 2024-05-09T08:56:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.661+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4b44079cf0 con 0x7f4b340039b0 2024-05-09T08:56:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.661+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400edd0 con 0x7f6f1c152e30 2024-05-09T08:56:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.662+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f14003990 con 0x7f6f1c152e30 2024-05-09T08:56:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.662+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14003d80 con 0x7f6f1c152e30 2024-05-09T08:56:09.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.662+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c1f0c40 con 0x7f6f1c152e30 2024-05-09T08:56:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.662+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1f1200 con 0x7f6f1c152e30 2024-05-09T08:56:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0125f0 con 0x7f560806dce0 2024-05-09T08:56:09.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc019070 con 0x7f560806dce0 2024-05-09T08:56:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56081457d0 con 0x7f560806dce0 2024-05-09T08:56:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5608145d90 con 0x7f560806dce0 2024-05-09T08:56:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f1c04f660 con 0x7f6f1c152e30 2024-05-09T08:56:09.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.663+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8810014030 con 0x7f8814045720 2024-05-09T08:56:09.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.664+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73ac044020 con 0x7f73bc06dac0 2024-05-09T08:56:09.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.664+0000 7f73a37fe640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 0x7f73a806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.664+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0129d0 con 0x7f560806dce0 2024-05-09T08:56:09.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.664+0000 7f73ba7fc640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 0x7f73a806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.665+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f73ac0abc80 con 0x7f73bc06dac0 2024-05-09T08:56:09.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.665+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f55fc012b70 con 0x7f560806dce0 2024-05-09T08:56:09.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.665+0000 7f73ba7fc640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 0x7f73a806e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73bc06c280 tx=0x7f73b0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.665+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56080c94e0 con 0x7f560806dce0 2024-05-09T08:56:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.665+0000 7f55effff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 0x7f55f406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.666+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f55fc08b210 con 0x7f560806dce0 2024-05-09T08:56:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.666+0000 7f5606ffd640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 0x7f55f406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.666+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73ac04b030 con 0x7f73bc06dac0 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f5606ffd640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 0x7f55f406e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55f8036000 tx=0x7f55f800c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004450 con 0x7f00dc06e000 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d4044070 con 0x7f00dc06e000 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004830 con 0x7f00dc06e000 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc159bc0 con 0x7f00dc06e000 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc0cbbb0 con 0x7f00dc06e000 2024-05-09T08:56:09.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.667+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ec4005a20 con 0x7f3ec404b090 2024-05-09T08:56:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.671+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00d4039040 con 0x7f00dc06e000 2024-05-09T08:56:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.670+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f1401b020 con 0x7f6f1c152e30 2024-05-09T08:56:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.670+0000 7f6f137fe640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.670+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f1408bd20 con 0x7f6f1c152e30 2024-05-09T08:56:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.671+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc154ee0 con 0x7f00dc06e000 2024-05-09T08:56:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.671+0000 7f00c37fe640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 0x7f00c806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.672+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f00d40ac2b0 con 0x7f00dc06e000 2024-05-09T08:56:09.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.672+0000 7f00da59c640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 0x7f00c806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.671+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56080c94e0 con 0x7f560806dce0 2024-05-09T08:56:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.671+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f1405a740 con 0x7f6f1c152e30 2024-05-09T08:56:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.672+0000 7f6f1affd640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:09.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.676+0000 7f00da59c640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 0x7f00c806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006f40 tx=0x7f00cc006e90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.679+0000 7f6f1affd640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 0x7f6efc06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009bd0 tx=0x7f6f0c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:09.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:09.681+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00dc154ee0 con 0x7f00dc06e000 2024-05-09T08:56:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.088+0000 7fdbce7fc640 1 -- 172.21.15.49:0/4118298801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdbd804f5f0 con 0x7fdbd81593e0 2024-05-09T08:56:10.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.088+0000 7fdbd4ff9640 1 -- 172.21.15.49:0/4118298801 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbd004b030 con 0x7fdbd81593e0 2024-05-09T08:56:10.088 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:10.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.091+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 msgr2=0x7fdbb806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.091+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 0x7fdbb806e3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006fd0 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 msgr2=0x7fdbd8157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009f70 tx=0x7fdbd0004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 shutdown_connections 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdbb806bf00 0x7fdbb806e3c0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0006fd0 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdbd81593e0 0x7fdbd8157730 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdbd80c6300 0x7fdbd81616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 --2- 172.21.15.49:0/4118298801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbd8070580 0x7fdbd80c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.092+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 >> 172.21.15.49:0/4118298801 conn(0x7fdbd8074840 msgr2=0x7fdbd80c3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.093+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 shutdown_connections 2024-05-09T08:56:10.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.094+0000 7fdbde4a5640 1 -- 172.21.15.49:0/4118298801 wait complete. 2024-05-09T08:56:10.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.094+0000 7f4b4ee33640 1 -- 172.21.15.49:0/3210334131 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4b40046f80 con 0x7f4b340039b0 2024-05-09T08:56:10.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.095+0000 7f4b3e7fc640 1 -- 172.21.15.49:0/3210334131 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b4407dff0 con 0x7f4b340039b0 2024-05-09T08:56:10.094 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:10.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.098+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 msgr2=0x7f4b2c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.098+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 0x7f4b2c06e3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b48065110 tx=0x7f4b48062d80 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 msgr2=0x7f4b40036880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b40036880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005770 tx=0x7f4b44004730 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 shutdown_connections 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4b2c06bf30 0x7f4b2c06e3f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b400354d0 0x7f4b400e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4b40034210 0x7f4b40034f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 --2- 172.21.15.49:0/3210334131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b340039b0 0x7f4b40036880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 >> 172.21.15.49:0/3210334131 conn(0x7f4b40051310 msgr2=0x7f4b400dacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.099+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 shutdown_connections 2024-05-09T08:56:10.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.100+0000 7f4b2bfff640 1 -- 172.21.15.49:0/3210334131 wait complete. 2024-05-09T08:56:10.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.105+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f87fc005580 con 0x7f8814045720 2024-05-09T08:56:10.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.105+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6f1c0685f0 con 0x7f6f1c152e30 2024-05-09T08:56:10.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.105+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3ec4005390 con 0x7f3ec404b090 2024-05-09T08:56:10.105 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-05-09T08:56:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.105+0000 7f8818ff9640 1 -- 172.21.15.49:0/2653289468 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f88100590b0 con 0x7f8814045720 2024-05-09T08:56:10.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.105+0000 7f6f137fe640 1 -- 172.21.15.49:0/1312239112 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6f1405a040 con 0x7f6f1c152e30 2024-05-09T08:56:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:10.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.106+0000 7f3eb7fff640 1 -- 172.21.15.49:0/3053948619 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ec40cb8e0 con 0x7f3ec404b090 2024-05-09T08:56:10.106 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-05-09T08:56:10.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.106+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f560804f5f0 con 0x7f560806dce0 2024-05-09T08:56:10.107 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-05-09T08:56:10.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.107+0000 7f55effff640 1 -- 172.21.15.49:0/2935518195 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55fc0594c0 con 0x7f560806dce0 2024-05-09T08:56:10.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.108+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 msgr2=0x7f6efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 0x7f6efc06e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009bd0 tx=0x7f6f0c009240 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 msgr2=0x7f6f1c1f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 0x7f6f1c1f32c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f14007d40 tx=0x7f6f1400e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 shutdown_connections 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f1c152e30 0x7f6f1c1f32c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f1c0c7130 0x7f6f1c148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 --2- 172.21.15.49:0/1312239112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f1c0c6450 0x7f6f1c1f0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 >> 172.21.15.49:0/1312239112 conn(0x7f6f1c0cc290 msgr2=0x7f6f1c0c3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.109+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f739c005850 con 0x7f73bc06dac0 2024-05-09T08:56:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 shutdown_connections 2024-05-09T08:56:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f6f21763640 1 -- 172.21.15.49:0/1312239112 wait complete. 2024-05-09T08:56:10.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f73a37fe640 1 -- 172.21.15.49:0/266266096 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f73ac07a610 con 0x7f73bc06dac0 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 msgr2=0x7f3eb006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 0x7f3eb006e3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc050ef0 tx=0x7f3ecc06a510 comp rx=0 tx=0).stop 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 msgr2=0x7f3ec4049d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3ec002a8b0 tx=0x7f3ec00046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 msgr2=0x7f880006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 0x7f880006e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8814045c30 tx=0x7f880802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 msgr2=0x7f88140ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f88140ce1d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f881000c850 tx=0x7f881000cd10 comp rx=0 tx=0).stop 2024-05-09T08:56:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.110+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 msgr2=0x7f55f406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 0x7f55f406e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f55f8036000 tx=0x7f55f800c040 comp rx=0 tx=0).stop 2024-05-09T08:56:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 shutdown_connections 2024-05-09T08:56:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 msgr2=0x7f56081ed160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 0x7f56081ed160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00d5f0 tx=0x7f55fc00dab0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f880006bf00 0x7f880006e3c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88140de3b0 0x7f88140d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f88140d5090 0x7f881417a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 --2- 172.21.15.49:0/2653289468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8814045720 0x7f88140ce1d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 >> 172.21.15.49:0/2653289468 conn(0x7f88140505d0 msgr2=0x7f881404e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 shutdown_connections 2024-05-09T08:56:10.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 shutdown_connections 2024-05-09T08:56:10.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3eb006bf00 0x7f3eb006e3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f406bf00 0x7f55f406e3c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56080c7030 0x7f56080cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560806e0c0 0x7f5608159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3ec40d4f90 0x7f3ec40c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3ec404b090 0x7f3ec4049d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 --2- 172.21.15.49:0/2935518195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560806dce0 0x7f56081ed160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 >> 172.21.15.49:0/2935518195 conn(0x7f5608067b50 msgr2=0x7f56080c7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 --2- 172.21.15.49:0/3053948619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ec400e2a0 0x7f3ec40497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 >> 172.21.15.49:0/3053948619 conn(0x7f3ec4050180 msgr2=0x7f3ec40cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 shutdown_connections 2024-05-09T08:56:10.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f8821599640 1 -- 172.21.15.49:0/2653289468 wait complete. 2024-05-09T08:56:10.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 shutdown_connections 2024-05-09T08:56:10.117 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:10.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.111+0000 7f560cdff640 1 -- 172.21.15.49:0/2935518195 wait complete. 2024-05-09T08:56:10.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-05-09T08:56:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.112+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 shutdown_connections 2024-05-09T08:56:10.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.112+0000 7f3ed1d14640 1 -- 172.21.15.49:0/3053948619 wait complete. 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.114+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f00dc04f660 con 0x7f00dc06e000 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.114+0000 7f00c37fe640 1 -- 172.21.15.49:0/636957257 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00dc0ca090 con 0x7f00dc06e000 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 msgr2=0x7f73a806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 0x7f73a806e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f73bc06c280 tx=0x7f73b0009190 comp rx=0 tx=0).stop 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 msgr2=0x7f73bc13ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 0x7f73bc13ed40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f73ac031000 tx=0x7f73ac0046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 shutdown_connections 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f73a806bf30 0x7f73a806e3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f73bc0ccdb0 0x7f73bc14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73bc0c7910 0x7f73bc159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 --2- 172.21.15.49:0/266266096 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f73bc06dac0 0x7f73bc13ed40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 >> 172.21.15.49:0/266266096 conn(0x7f73bc0cc290 msgr2=0x7f73bc134bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 shutdown_connections 2024-05-09T08:56:10.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.115+0000 7f73c155a640 1 -- 172.21.15.49:0/266266096 wait complete. 2024-05-09T08:56:10.121 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-05-09T08:56:10.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 msgr2=0x7f00c806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 0x7f00c806e3c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f00cc006f40 tx=0x7f00cc006e90 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 msgr2=0x7f00dc1632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009d90 tx=0x7f00d4004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 shutdown_connections 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f00c806bf00 0x7f00c806e3c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00dc151000 0x7f00dc1359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00dc0c7910 0x7f00dc1f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 --2- 172.21.15.49:0/636957257 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f00dc06e000 0x7f00dc1632c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 >> 172.21.15.49:0/636957257 conn(0x7f00dc0cc290 msgr2=0x7f00dc0c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:10.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.118+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 shutdown_connections 2024-05-09T08:56:10.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:10.119+0000 7f00e1170640 1 -- 172.21.15.49:0/636957257 wait complete. 2024-05-09T08:56:10.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-05-09T08:56:10.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-05-09T08:56:10.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-05-09T08:56:10.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-05-09T08:56:10.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-09T08:56:10.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-05-09T08:56:10.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:56:10.323 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:56:11.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:56:11.123 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:56:11.134 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:56:11.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:56:11.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:56:11.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:56:11.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:56:11.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.286+0000 7f93b2e19640 1 Processor -- start 2024-05-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.286+0000 7f93b2e19640 1 -- start start 2024-05-09T08:56:11.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.288+0000 7f93b2e19640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93a404e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.288+0000 7f93b2e19640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93a40476c0 0x7f93a404edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.288+0000 7f93b2e19640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a400abf0 con 0x7f93a404f2f0 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.288+0000 7f93b2e19640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f93a404d060 con 0x7f93a40476c0 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.288+0000 7f93b2e19640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93a404d820 con 0x7f93a40d7090 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.289+0000 7f93b2618640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f93a404f2f0 0x7f93a40e0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:36432/0 (socket says 172.21.15.49:36432) 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.289+0000 7f93b2618640 1 -- 172.21.15.49:0/3445250707 learned_addr learned my addr 172.21.15.49:0/3445250707 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.289+0000 7f93b1616640 1 --2- 172.21.15.49:0/3445250707 >> v2:172.21.15.183:3300/0 conn(0x7f93a40476c0 0x7f93a404edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.289+0000 7f93b1e17640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93a404e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1338216673 0 0) 0x7f93a400abf0 con 0x7f93a404f2f0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9398003520 con 0x7f93a404f2f0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3591323871 0 0) 0x7f9398003520 con 0x7f93a404f2f0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 msgr2=0x7f93a404e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93a404e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 >> v2:172.21.15.183:3300/0 conn(0x7f93a40476c0 msgr2=0x7f93a404edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 --2- 172.21.15.49:0/3445250707 >> v2:172.21.15.183:3300/0 conn(0x7f93a40476c0 0x7f93a404edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.290+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a404db00 con 0x7f93a404f2f0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f93ac06a1d0 con 0x7f93a404f2f0 2024-05-09T08:56:11.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 >> v1:172.21.15.183:6789/0 conn(0x7f93a404f2f0 legacy=0x7f93a40e0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9398003f30 0x7f9398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9398004d40 0x7f93980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93a404d060 con 0x7f9398003f30 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93a404d820 con 0x7f9398004d40 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.291+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f939800b9b0 con 0x7f93a40d7090 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9398004d40 0x7f93980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b1e17640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b1616640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9398003f30 0x7f9398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 -- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 msgr2=0x7f93980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a40d7090 0x7f93980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 -- 172.21.15.49:0/3445250707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9398003f30 msgr2=0x7f9398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9398003f30 0x7f9398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.292+0000 7f93b2618640 1 -- 172.21.15.49:0/3445250707 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9398003520 con 0x7f9398004d40 2024-05-09T08:56:11.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.298+0000 7f93b2618640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9398004d40 0x7f93980051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0503a0 tx=0x7f9398005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59db78f14855ed14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.298+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0b4070 con 0x7f9398004d40 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.298+0000 7f93b0e15640 1 -- 172.21.15.49:0/3445250707 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac0a7d70 con 0x7f9398004d40 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.299+0000 7f93b2e19640 1 -- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9398004d40 msgr2=0x7f93980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.299+0000 7f93b2e19640 1 --2- 172.21.15.49:0/3445250707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9398004d40 0x7f93980051c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0503a0 tx=0x7f9398005830 comp rx=0 tx=0).stop 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.299+0000 7f93b2618640 1 -- 172.21.15.49:0/3445250707 reap_dead start 2024-05-09T08:56:11.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.299+0000 7f93b2e19640 1 -- 172.21.15.49:0/3445250707 shutdown_connections 2024-05-09T08:56:11.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.299+0000 7f93b2e19640 1 -- 172.21.15.49:0/3445250707 >> 172.21.15.49:0/3445250707 conn(0x7f93a400e6a0 msgr2=0x7f93a4009d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.300+0000 7f93b2e19640 1 -- 172.21.15.49:0/3445250707 shutdown_connections 2024-05-09T08:56:11.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.300+0000 7f93b2e19640 1 -- 172.21.15.49:0/3445250707 wait complete. 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.302+0000 7f93b2e19640 1 Processor -- start 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.302+0000 7f93b2e19640 1 -- start start 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 0x7f93a4052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 0x7f93a4052890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 0x7f93a4195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93a40e0840 con 0x7f93a404c600 2024-05-09T08:56:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93a404cd20 con 0x7f93a40d5b30 2024-05-09T08:56:11.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.303+0000 7f93b2e19640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93a40e09c0 con 0x7f93a404a9e0 2024-05-09T08:56:11.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 0x7f93a4052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1616640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 0x7f93a4052890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1616640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 0x7f93a4052890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34372/0 (socket says 172.21.15.49:34372) 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1616640 1 -- 172.21.15.49:0/940892369 learned_addr learned my addr 172.21.15.49:0/940892369 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b2618640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 0x7f93a4195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 -- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 msgr2=0x7f93a4052890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 0x7f93a4052890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 -- 172.21.15.49:0/940892369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 msgr2=0x7f93a4195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 0x7f93a4195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b1e17640 1 -- 172.21.15.49:0/940892369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a404ae00 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.304+0000 7f93b2618640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 0x7f93a4195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.305+0000 7f93b1e17640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 0x7f93a4052350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f939c011960 tx=0x7f939c011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.305+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c019070 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.305+0000 7f93b2e19640 1 -- 172.21.15.49:0/940892369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a40da550 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.305+0000 7f93b2e19640 1 -- 172.21.15.49:0/940892369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a40dd280 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.306+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939c004300 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.306+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939c00fa40 con 0x7f93a40d5b30 2024-05-09T08:56:11.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.310+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f939c01c070 con 0x7f93a40d5b30 2024-05-09T08:56:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.311+0000 7f93a27fc640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 0x7f939806f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.311+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f939c08b2c0 con 0x7f93a40d5b30 2024-05-09T08:56:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.312+0000 7f93b1616640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 0x7f939806f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.312+0000 7f93b2e19640 1 -- 172.21.15.49:0/940892369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394005290 con 0x7f93a40d5b30 2024-05-09T08:56:11.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.313+0000 7f93b1616640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 0x7f939806f450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93a40df7b0 tx=0x7f93a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.318+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f939c08b0c0 con 0x7f93a40d5b30 2024-05-09T08:56:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.340+0000 7f5ae4a21640 1 Processor -- start 2024-05-09T08:56:11.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.340+0000 7f5ae4a21640 1 -- start start 2024-05-09T08:56:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.341+0000 7f5ae4a21640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 0x7f5ad804b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.341+0000 7f5ae4a21640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.341+0000 7f5ae4a21640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad80dd850 con 0x7f5ad80e66a0 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.341+0000 7f5ae4a21640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad80e7610 con 0x7f5ad80dd470 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.341+0000 7f5ae4a21640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad8007d70 con 0x7f5ad8006280 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34384/0 (socket says 172.21.15.49:34384) 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 -- 172.21.15.49:0/3724896685 learned_addr learned my addr 172.21.15.49:0/3724896685 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adf7fe640 1 --2- 172.21.15.49:0/3724896685 >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 0x7f5ad804b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 -- 172.21.15.49:0/3724896685 >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 msgr2=0x7f5ad804b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 --2- 172.21.15.49:0/3724896685 >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 0x7f5ad804b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.342+0000 7f5adeffd640 1 -- 172.21.15.49:0/3724896685 >> v1:172.21.15.49:6789/0 conn(0x7f5ad80e66a0 legacy=0x7f5ad80e6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.343+0000 7f5adeffd640 1 -- 172.21.15.49:0/3724896685 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad8007160 con 0x7f5ad8006280 2024-05-09T08:56:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.344+0000 7f5adf7fe640 1 --2- 172.21.15.49:0/3724896685 >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 0x7f5ad804b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.344+0000 7f5adeffd640 1 --2- 172.21.15.49:0/3724896685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f3c11df24887eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.344+0000 7f5ade7fc640 1 -- 172.21.15.49:0/3724896685 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4037070 con 0x7f5ad8006280 2024-05-09T08:56:11.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.345+0000 7f5ade7fc640 1 -- 172.21.15.49:0/3724896685 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4032440 con 0x7f5ad8006280 2024-05-09T08:56:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.345+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3724896685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 msgr2=0x7f5ad80066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.345+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3724896685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad402a230 comp rx=0 tx=0).stop 2024-05-09T08:56:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3724896685 shutdown_connections 2024-05-09T08:56:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3724896685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad8006280 0x7f5ad80066a0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad402a230 comp rx=0 tx=0).stop 2024-05-09T08:56:11.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3724896685 >> v2:172.21.15.183:3300/0 conn(0x7f5ad80dd470 0x7f5ad804b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f8b093db640 1 Processor -- start 2024-05-09T08:56:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3724896685 >> 172.21.15.49:0/3724896685 conn(0x7f5ad8050a10 msgr2=0x7f5ad8050e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.346+0000 7f8b093db640 1 -- start start 2024-05-09T08:56:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3724896685 shutdown_connections 2024-05-09T08:56:11.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3724896685 wait complete. 2024-05-09T08:56:11.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040ccdb0 0x7f8b040c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8b040c7910 0x7f8b04150d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b041512d0 0x7f8b0406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8b041571a0 con 0x7f8b041512d0 2024-05-09T08:56:11.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8b04157480 con 0x7f8b040c7910 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.347+0000 7f8b093db640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8b04157760 con 0x7f8b040ccdb0 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.348+0000 7f8b037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b041512d0 0x7f8b0406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.348+0000 7f8b037fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8b041512d0 0x7f8b0406dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48260/0 (socket says 172.21.15.49:48260) 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.348+0000 7f8b037fe640 1 -- 172.21.15.49:0/3844296451 learned_addr learned my addr 172.21.15.49:0/3844296451 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.348+0000 7f8b02ffd640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040ccdb0 0x7f8b040c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7ff63ffff640 1 Processor -- start 2024-05-09T08:56:11.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b027fc640 1 --2- 172.21.15.49:0/3844296451 >> v2:172.21.15.183:3300/0 conn(0x7f8b040c7910 0x7f8b04150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f11b13b0640 1 Processor -- start 2024-05-09T08:56:11.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7ff63ffff640 1 -- start start 2024-05-09T08:56:11.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7fe954d32640 1 Processor -- start 2024-05-09T08:56:11.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7fe954d32640 1 -- start start 2024-05-09T08:56:11.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f5ae4a21640 1 Processor -- start 2024-05-09T08:56:11.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f2b4bfff640 1 Processor -- start 2024-05-09T08:56:11.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b037fe640 1 -- 172.21.15.49:0/3844296451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040ccdb0 msgr2=0x7f8b040c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f5ae4a21640 1 -- start start 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b037fe640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040ccdb0 0x7f8b040c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b037fe640 1 -- 172.21.15.49:0/3844296451 >> v2:172.21.15.183:3300/0 conn(0x7f8b040c7910 msgr2=0x7f8b04150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b037fe640 1 --2- 172.21.15.49:0/3844296451 >> v2:172.21.15.183:3300/0 conn(0x7f8b040c7910 0x7f8b04150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f8b037fe640 1 -- 172.21.15.49:0/3844296451 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b04157a40 con 0x7f8b041512d0 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.349+0000 7f11b13b0640 1 -- start start 2024-05-09T08:56:11.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11b13b0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f11a40dac60 0x7f11a404b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff640156d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff640068560 0x7ff640150d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 0x7ff64006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff6400c70a0 con 0x7ff6400c3450 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff640070d20 con 0x7ff640068560 2024-05-09T08:56:11.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63ffff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff6400c75d0 con 0x7ff640071770 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11b13b0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a4050ca0 0x7f11a4034a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11b13b0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a4007710 con 0x7f11a404b500 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11b13b0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f11a40068b0 con 0x7f11a40dac60 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11b13b0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f11a4006b90 con 0x7f11a4050ca0 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11abfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f11a404b500 0x7f11a4005e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46010/0 (socket says 172.21.15.49:46010) 2024-05-09T08:56:11.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe954d32640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe950155060 0x7fe9500c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7ff63effd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff640156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7ff63e7fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff640068560 0x7ff640150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.350+0000 7f11abfff640 1 -- 172.21.15.49:0/89239050 learned_addr learned my addr 172.21.15.49:0/89239050 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:11.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11b0baf640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a4050ca0 0x7f11a4034a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f8b037fe640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b041512d0 0x7f8b0406dac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cd2b0 tx=0x7f8af802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71271e63ba15da4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7ff63e7fc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff640068560 0x7ff640150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55624/0 (socket says 172.21.15.49:55624) 2024-05-09T08:56:11.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7ff63e7fc640 1 -- 172.21.15.49:0/276935857 learned_addr learned my addr 172.21.15.49:0/276935857 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11ab7fe640 1 --2- 172.21.15.49:0/89239050 >> v2:172.21.15.183:3300/0 conn(0x7f11a40dac60 0x7f11a404b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f2b4bfff640 1 -- start start 2024-05-09T08:56:11.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe94e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe950155060 0x7fe9500c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe94e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe950155060 0x7fe9500c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48286/0 (socket says 172.21.15.49:48286) 2024-05-09T08:56:11.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe94e59c640 1 -- 172.21.15.49:0/805554747 learned_addr learned my addr 172.21.15.49:0/805554747 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe954d32640 1 --2- 172.21.15.49:0/805554747 >> v2:172.21.15.183:3300/0 conn(0x7fe95006e480 0x7fe95006b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11ab7fe640 1 -- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a4050ca0 msgr2=0x7f11a4034a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11aaffd640 1 -- 172.21.15.49:0/89239050 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893815413 0 0) 0x7f11a4007710 con 0x7f11a404b500 2024-05-09T08:56:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe950155440 con 0x7fe95006bdd0 2024-05-09T08:56:11.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe95006c200 con 0x7fe950155060 2024-05-09T08:56:11.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11ab7fe640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a4050ca0 0x7f11a4034a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11ab7fe640 1 -- 172.21.15.49:0/89239050 >> v1:172.21.15.49:6789/0 conn(0x7f11a404b500 legacy=0x7f11a4005e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7f11ab7fe640 1 -- 172.21.15.49:0/89239050 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a4006e70 con 0x7f11a40dac60 2024-05-09T08:56:11.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5ae4a21640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ad80dd470 0x7f5ad8048770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe95015e3f0 con 0x7fe95006e480 2024-05-09T08:56:11.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f8b01ffb640 1 -- 172.21.15.49:0/3844296451 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8037070 con 0x7f8b041512d0 2024-05-09T08:56:11.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f8b01ffb640 1 -- 172.21.15.49:0/3844296451 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af8032440 con 0x7f8b041512d0 2024-05-09T08:56:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5ae4a21640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5ae4a21640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 0x7f5ad817a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f2b4bfff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2b4c0ccbe0 0x7f2b4c14d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5adeffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34422/0 (socket says 172.21.15.49:34422) 2024-05-09T08:56:11.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f2b4bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b4c06e140 0x7f2b4c06e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f2b4bfff640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4c0c6c70 con 0x7f2b4c157790 2024-05-09T08:56:11.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5adeffd640 1 -- 172.21.15.49:0/3175175611 learned_addr learned my addr 172.21.15.49:0/3175175611 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8007a00 con 0x7f5ad8049500 2024-05-09T08:56:11.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad8007720 con 0x7f5ad80dd470 2024-05-09T08:56:11.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad804b8d0 con 0x7f5ad80e66a0 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7fe94e59c640 1 -- 172.21.15.49:0/805554747 >> v2:172.21.15.183:3300/0 conn(0x7fe95006e480 msgr2=0x7fe95006b890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7fe94e59c640 1 --2- 172.21.15.49:0/805554747 >> v2:172.21.15.183:3300/0 conn(0x7fe95006e480 0x7fe95006b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7fe94e59c640 1 -- 172.21.15.49:0/805554747 >> v1:172.21.15.183:6789/0 conn(0x7fe95006bdd0 legacy=0x7fe9500c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7fe94e59c640 1 -- 172.21.15.49:0/805554747 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95015e6d0 con 0x7fe950155060 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f2b4bfff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c0c6f50 con 0x7f2b4c06e140 2024-05-09T08:56:11.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7f2b4bfff640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c0c7230 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 -- 172.21.15.49:0/3844296451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b041512d0 msgr2=0x7f8b0406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b041512d0 0x7f8b0406dac0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cd2b0 tx=0x7f8af802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 -- 172.21.15.49:0/3844296451 shutdown_connections 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b041512d0 0x7f8b0406dac0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 --2- 172.21.15.49:0/3844296451 >> v2:172.21.15.183:3300/0 conn(0x7f8b040c7910 0x7f8b04150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 --2- 172.21.15.49:0/3844296451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040ccdb0 0x7f8b040c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 -- 172.21.15.49:0/3844296451 >> 172.21.15.49:0/3844296451 conn(0x7f8b040cc290 msgr2=0x7f8b040cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.351+0000 7ff63e7fc640 1 -- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 msgr2=0x7ff640156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7ff63f7fe640 1 --2- 172.21.15.49:0/276935857 >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 0x7ff64006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff640156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7ff63e7fc640 1 -- 172.21.15.49:0/276935857 >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 msgr2=0x7ff64006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/276935857 >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 0x7ff64006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.352+0000 7ff63e7fc640 1 -- 172.21.15.49:0/276935857 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff640070f70 con 0x7ff640068560 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7ff63f7fe640 1 --2- 172.21.15.49:0/276935857 >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 0x7ff64006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:11.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7ff63effd640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff640156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:11.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adf7fe640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ad80dd470 0x7f5ad8048770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f11ab7fe640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a40dac60 0x7f11a404b8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f11980097a0 tx=0x7f119802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbbaccab7abe4d8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adffff640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 0x7f5ad817a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f11aaffd640 1 -- 172.21.15.49:0/89239050 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198037070 con 0x7f11a40dac60 2024-05-09T08:56:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f11aaffd640 1 -- 172.21.15.49:0/89239050 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1198032440 con 0x7f11a40dac60 2024-05-09T08:56:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f11aaffd640 1 -- 172.21.15.49:0/89239050 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198032810 con 0x7f11a40dac60 2024-05-09T08:56:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adeffd640 1 -- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ad80dd470 msgr2=0x7f5ad8048770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff640068560 0x7ff640150d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff64006efc0 tx=0x7ff63802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bf7db952b1b109 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7fe94e59c640 1 --2- 172.21.15.49:0/805554747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe9500c6c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9500c71d0 tx=0x7fe94402a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8399fa2655861384 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f8b093db640 1 -- 172.21.15.49:0/3844296451 shutdown_connections 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f2b4a7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2b4c157790 0x7f2b4c0c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46012/0 (socket says 172.21.15.49:46012) 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f2b4a7fc640 1 -- 172.21.15.49:0/247532773 learned_addr learned my addr 172.21.15.49:0/247532773 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f2b4b7fe640 1 --2- 172.21.15.49:0/247532773 >> v2:172.21.15.49:3300/0 conn(0x7f2b4c06e140 0x7f2b4c06e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7ff63dffb640 1 -- 172.21.15.49:0/276935857 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638037070 con 0x7ff640068560 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7ff63dffb640 1 -- 172.21.15.49:0/276935857 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff638032440 con 0x7ff640068560 2024-05-09T08:56:11.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7ff63dffb640 1 -- 172.21.15.49:0/276935857 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638037070 con 0x7ff640068560 2024-05-09T08:56:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f2b4affd640 1 --2- 172.21.15.49:0/247532773 >> v2:172.21.15.183:3300/0 conn(0x7f2b4c0ccbe0 0x7f2b4c14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adeffd640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ad80dd470 0x7f5ad8048770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adeffd640 1 -- 172.21.15.49:0/3175175611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 msgr2=0x7f5ad817a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adeffd640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 0x7f5ad817a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f8b093db640 1 -- 172.21.15.49:0/3844296451 wait complete. 2024-05-09T08:56:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adeffd640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad8036a50 con 0x7f5ad80e66a0 2024-05-09T08:56:11.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.353+0000 7f5adffff640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 0x7f5ad817a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 -- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a40dac60 msgr2=0x7f11a404b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a40dac60 0x7f11a404b8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f11980097a0 tx=0x7f119802a210 comp rx=0 tx=0).stop 2024-05-09T08:56:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7fe94d59a640 1 -- 172.21.15.49:0/805554747 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944036070 con 0x7fe950155060 2024-05-09T08:56:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7fe94d59a640 1 -- 172.21.15.49:0/805554747 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944031600 con 0x7fe950155060 2024-05-09T08:56:11.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7fe94d59a640 1 -- 172.21.15.49:0/805554747 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944031a10 con 0x7fe950155060 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 -- 172.21.15.49:0/89239050 shutdown_connections 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a4050ca0 0x7f11a4034a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 --2- 172.21.15.49:0/89239050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a40dac60 0x7f11a404b8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 -- 172.21.15.49:0/89239050 >> 172.21.15.49:0/89239050 conn(0x7f11a4050180 msgr2=0x7f11a4050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f2b4b7fe640 1 -- 172.21.15.49:0/247532773 >> v2:172.21.15.183:3300/0 conn(0x7f2b4c0ccbe0 msgr2=0x7f2b4c14d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f2b4b7fe640 1 --2- 172.21.15.49:0/247532773 >> v2:172.21.15.183:3300/0 conn(0x7f2b4c0ccbe0 0x7f2b4c14d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f2b4b7fe640 1 -- 172.21.15.49:0/247532773 >> v1:172.21.15.49:6789/0 conn(0x7f2b4c157790 legacy=0x7f2b4c0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f2b4b7fe640 1 -- 172.21.15.49:0/247532773 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c0c7510 con 0x7f2b4c06e140 2024-05-09T08:56:11.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 -- 172.21.15.49:0/89239050 shutdown_connections 2024-05-09T08:56:11.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f11b13b0640 1 -- 172.21.15.49:0/89239050 wait complete. 2024-05-09T08:56:11.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.354+0000 7f5adeffd640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f336098c640 1 Processor -- start 2024-05-09T08:56:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f336098c640 1 -- start start 2024-05-09T08:56:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 -- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff640068560 msgr2=0x7ff640150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff640068560 0x7ff640150d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff64006efc0 tx=0x7ff63802a2a0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 msgr2=0x7fe9500c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4040020 con 0x7f5ad80e66a0 2024-05-09T08:56:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f2b4b7fe640 1 --2- 172.21.15.49:0/247532773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c06e140 0x7f2b4c06e550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06bef0 tx=0x7f2b4402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a5fe330b0fe77f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f2b49ffb640 1 -- 172.21.15.49:0/247532773 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007740 con 0x7f2b4c06e140 2024-05-09T08:56:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f2b49ffb640 1 -- 172.21.15.49:0/247532773 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b440078e0 con 0x7f2b4c06e140 2024-05-09T08:56:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7fe954d32640 1 --2- 172.21.15.49:0/805554747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe9500c6c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9500c71d0 tx=0x7fe94402a5a0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f8b093db640 1 Processor -- start 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4038040 con 0x7f5ad80e66a0 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4037070 con 0x7f5ad80e66a0 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad8048cb0 con 0x7f5ad80e66a0 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f8b093db640 1 -- start start 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f11b13b0640 1 Processor -- start 2024-05-09T08:56:11.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 shutdown_connections 2024-05-09T08:56:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f336098c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 0x7f335404d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7fe954d32640 1 --2- 172.21.15.49:0/805554747 >> v2:172.21.15.183:3300/0 conn(0x7fe95006e480 0x7fe95006b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f2b49ffb640 1 -- 172.21.15.49:0/247532773 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007cb0 con 0x7f2b4c06e140 2024-05-09T08:56:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7fe954d32640 1 --2- 172.21.15.49:0/805554747 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe9500c6c90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.356+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad8049c90 con 0x7f5ad80e66a0 2024-05-09T08:56:11.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 >> 172.21.15.49:0/805554747 conn(0x7fe9500cc290 msgr2=0x7fe9500cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f335b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 0x7f335404d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f335b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 0x7f335404d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48304/0 (socket says 172.21.15.49:48304) 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f335affd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f33540474e0 0x7f3354047930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46022/0 (socket says 172.21.15.49:46022) 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f336098c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 0x7f33540e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f335affd640 1 -- 172.21.15.49:0/542674534 learned_addr learned my addr 172.21.15.49:0/542674534 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:11.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 shutdown_connections 2024-05-09T08:56:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f335bfff640 1 --2- 172.21.15.49:0/542674534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 0x7f33540e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f335404da50 con 0x7f33540474e0 2024-05-09T08:56:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7fe954d32640 1 -- 172.21.15.49:0/805554747 wait complete. 2024-05-09T08:56:11.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f11b13b0640 1 -- start start 2024-05-09T08:56:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3354007650 con 0x7f33540d6e50 2024-05-09T08:56:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3354006a40 con 0x7f33540e0170 2024-05-09T08:56:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7f8b093db640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 0x7f11a4045340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b093db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b040ccdb0 0x7f8b040cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335a7fc640 1 -- 172.21.15.49:0/542674534 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465490233 0 0) 0x7f335404da50 con 0x7f33540474e0 2024-05-09T08:56:11.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b093db640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 0x7f8b04136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335a7fc640 1 -- 172.21.15.49:0/542674534 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f334c003520 con 0x7f33540474e0 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a4050ca0 0x7f11a40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 0x7f11a4184f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40065a0 con 0x7f11a4050ca0 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11a4007430 con 0x7f11a404b500 2024-05-09T08:56:11.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b13b0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40d5000 con 0x7f11a40dac60 2024-05-09T08:56:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b093db640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8b04156dd0 con 0x7f8b0414eaf0 2024-05-09T08:56:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b093db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040c71d0 con 0x7f8b040ccdb0 2024-05-09T08:56:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 0x7f11a4045340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11b0baf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 0x7f11a4184f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 0x7f11a4045340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55678/0 (socket says 172.21.15.49:55678) 2024-05-09T08:56:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f11abfff640 1 -- 172.21.15.49:0/654466039 learned_addr learned my addr 172.21.15.49:0/654466039 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b093db640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8b04156f50 con 0x7f8b040c7910 2024-05-09T08:56:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad80057e0 con 0x7f5ad80e66a0 2024-05-09T08:56:11.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b02ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 -- 172.21.15.49:0/276935857 shutdown_connections 2024-05-09T08:56:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 --2- 172.21.15.49:0/276935857 >> v2:172.21.15.49:3300/0 conn(0x7ff6400c3450 0x7ff64006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff640068560 0x7ff640150d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 --2- 172.21.15.49:0/276935857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff640156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 -- 172.21.15.49:0/276935857 >> 172.21.15.49:0/276935857 conn(0x7ff640074f20 msgr2=0x7ff6400cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 -- 172.21.15.49:0/276935857 shutdown_connections 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.355+0000 7ff63ffff640 1 -- 172.21.15.49:0/276935857 wait complete. 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7ff63ffff640 1 Processor -- start 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.357+0000 7ff63ffff640 1 -- start start 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff640068560 0x7ff6400c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6400c3450 0x7ff64012c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff640070560 con 0x7ff640068560 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff640157250 con 0x7ff6400c3450 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63ffff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff6400706e0 con 0x7ff640071770 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63e7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63e7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34444/0 (socket says 172.21.15.49:34444) 2024-05-09T08:56:11.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7ff63e7fc640 1 -- 172.21.15.49:0/4176065430 learned_addr learned my addr 172.21.15.49:0/4176065430 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b02ffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34430/0 (socket says 172.21.15.49:34430) 2024-05-09T08:56:11.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b02ffd640 1 -- 172.21.15.49:0/2966518054 learned_addr learned my addr 172.21.15.49:0/2966518054 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f8b037fe640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 0x7f8b04136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 -- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6400c3450 msgr2=0x7ff64012c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6400c3450 0x7ff64012c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 -- 172.21.15.49:0/4176065430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff640068560 msgr2=0x7ff6400c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff640068560 0x7ff6400c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 -- 172.21.15.49:0/4176065430 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64015a540 con 0x7ff640071770 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff63e7fc640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff63802a2e0 tx=0x7ff6380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335bfff640 1 -- 172.21.15.49:0/542674534 >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 msgr2=0x7f335404d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335bfff640 1 --2- 172.21.15.49:0/542674534 >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 0x7f335404d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335bfff640 1 -- 172.21.15.49:0/542674534 >> v1:172.21.15.49:6789/0 conn(0x7f33540474e0 legacy=0x7f3354047930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.358+0000 7f335bfff640 1 -- 172.21.15.49:0/542674534 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3354006d20 con 0x7f33540e0170 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 -- 172.21.15.49:0/247532773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c06e140 msgr2=0x7f2b4c06e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/247532773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c06e140 0x7f2b4c06e550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c06bef0 tx=0x7f2b4402a710 comp rx=0 tx=0).stop 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 -- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 msgr2=0x7f11a4045340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 0x7f11a4045340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 -- 172.21.15.49:0/654466039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a4050ca0 msgr2=0x7f11a40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a4050ca0 0x7f11a40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 -- 172.21.15.49:0/654466039 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40ca690 con 0x7f11a40dac60 2024-05-09T08:56:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 -- 172.21.15.49:0/247532773 shutdown_connections 2024-05-09T08:56:11.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f335bfff640 1 --2- 172.21.15.49:0/542674534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 0x7f33540e0600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f335c069ec0 tx=0x7f335c0aafc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503eb06746654d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/247532773 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c06e140 0x7f2b4c06e550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/247532773 >> v2:172.21.15.183:3300/0 conn(0x7f2b4c0ccbe0 0x7f2b4c14d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 -- 172.21.15.49:0/247532773 >> 172.21.15.49:0/247532773 conn(0x7f2b4c0cc0f0 msgr2=0x7f2b4c0cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638042070 con 0x7ff640071770 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff638030750 con 0x7ff640071770 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638037070 con 0x7ff640071770 2024-05-09T08:56:11.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f11b0baf640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 0x7f11a4184f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11ac050ef0 tx=0x7f11ac068930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f8b027fc640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b040ccdb0 0x7f8b040cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7fe954d32640 1 Processor -- start 2024-05-09T08:56:11.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f335a7fc640 1 -- 172.21.15.49:0/542674534 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0b8070 con 0x7f33540e0170 2024-05-09T08:56:11.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f335a7fc640 1 -- 172.21.15.49:0/542674534 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c0abe10 con 0x7f33540e0170 2024-05-09T08:56:11.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f335a7fc640 1 -- 172.21.15.49:0/542674534 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0b3640 con 0x7f33540e0170 2024-05-09T08:56:11.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7fe954d32640 1 -- start start 2024-05-09T08:56:11.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.359+0000 7f2b4bfff640 1 -- 172.21.15.49:0/247532773 shutdown_connections 2024-05-09T08:56:11.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f2b4bfff640 1 -- 172.21.15.49:0/247532773 wait complete. 2024-05-09T08:56:11.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b02ffd640 1 -- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b040ccdb0 msgr2=0x7f8b040cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b02ffd640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b040ccdb0 0x7f8b040cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b02ffd640 1 -- 172.21.15.49:0/2966518054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 msgr2=0x7f8b04136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b02ffd640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 0x7f8b04136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b02ffd640 1 -- 172.21.15.49:0/2966518054 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040ca590 con 0x7f8b040c7910 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f8b037fe640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 0x7f8b04136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64015a820 con 0x7ff640071770 2024-05-09T08:56:11.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64015ade0 con 0x7ff640071770 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac093030 con 0x7f11a40dac60 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac0736e0 con 0x7f11a40dac60 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a404e190 con 0x7f11a40dac60 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a404e750 con 0x7f11a40dac60 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ad4043020 con 0x7f5ad80e66a0 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.360+0000 7f5acbfff640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 0x7f5ac406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad40ab3a0 con 0x7f5ad80e66a0 2024-05-09T08:56:11.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 msgr2=0x7f33540e0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 --2- 172.21.15.49:0/542674534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 0x7f33540e0600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f335c069ec0 tx=0x7f335c0aafc0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f8b02ffd640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8af400e8c0 tx=0x7f8af400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 shutdown_connections 2024-05-09T08:56:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 --2- 172.21.15.49:0/542674534 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540e0170 0x7f33540e0600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 --2- 172.21.15.49:0/542674534 >> v2:172.21.15.49:3300/0 conn(0x7f33540d6e50 0x7f335404d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 >> 172.21.15.49:0/542674534 conn(0x7f3354052390 msgr2=0x7f33540527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f5adf7fe640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 0x7f5ac406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0697c0 con 0x7f11a40dac60 2024-05-09T08:56:11.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.361+0000 7f2b4bfff640 1 Processor -- start 2024-05-09T08:56:11.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 shutdown_connections 2024-05-09T08:56:11.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7f5adf7fe640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 0x7f5ac406e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00a990 tx=0x7f5acc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7f2b4bfff640 1 -- start start 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95006bdd0 0x7fe9501e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe95014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe95015ef70 con 0x7fe950155060 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe95015ec90 con 0x7fe95006bdd0 2024-05-09T08:56:11.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe954d32640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe9500c8570 con 0x7fe95006e480 2024-05-09T08:56:11.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7f336098c640 1 -- 172.21.15.49:0/542674534 wait complete. 2024-05-09T08:56:11.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34446/0 (socket says 172.21.15.49:34446) 2024-05-09T08:56:11.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 -- 172.21.15.49:0/2903882327 learned_addr learned my addr 172.21.15.49:0/2903882327 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64004f5f0 con 0x7ff640071770 2024-05-09T08:56:11.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118c005290 con 0x7f11a40dac60 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94ed9d640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe95014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94e59c640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95006bdd0 0x7fe9501e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 -- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95006bdd0 msgr2=0x7fe9501e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95006bdd0 0x7fe9501e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 -- 172.21.15.49:0/2903882327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 msgr2=0x7fe95014f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe95014f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.362+0000 7fe94dd9b640 1 -- 172.21.15.49:0/2903882327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9501e4190 con 0x7fe95006e480 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 0x7f2b4c147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 0x7f2b4c148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b4c157790 0x7f2b4c1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c14da20 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c14d8a0 con 0x7f2b4c06e140 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4bfff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c0c3d00 con 0x7f2b4c157790 2024-05-09T08:56:11.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7fe94dd9b640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b480 tx=0x7fe94000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 0x7f2b4c147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 0x7f2b4c148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 0x7f2b4c147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55718/0 (socket says 172.21.15.49:55718) 2024-05-09T08:56:11.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4affd640 1 -- 172.21.15.49:0/301596482 learned_addr learned my addr 172.21.15.49:0/301596482 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4b7fe640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b4c157790 0x7f2b4c1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 -- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b4c157790 msgr2=0x7f2b4c1461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b4c157790 0x7f2b4c1461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 -- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 msgr2=0x7f2b4c147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 0x7f2b4c147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 -- 172.21.15.49:0/301596482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c202070 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.363+0000 7f2b4a7fc640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 0x7f2b4c148a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c800 tx=0x7f2b3c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f336098c640 1 Processor -- start 2024-05-09T08:56:11.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c017070 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0058b0 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c005c60 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f336098c640 1 -- start start 2024-05-09T08:56:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c202330 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f336098c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f336098c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 0x7f3354170e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f336098c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 0x7f3354171350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.364+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c0cbd80 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:55726/0 (socket says 172.21.15.49:55726) 2024-05-09T08:56:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 0x7f3354170e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5ad404a030 con 0x7f5ad80e66a0 2024-05-09T08:56:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 0x7f3354170e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34464/0 (socket says 172.21.15.49:34464) 2024-05-09T08:56:11.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335b7fe640 1 -- 172.21.15.49:0/616843902 learned_addr learned my addr 172.21.15.49:0/616843902 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f33540d7230 con 0x7f33540e0170 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33540072e0 con 0x7f33540474e0 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f335404e910 con 0x7f33540d6e50 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 0x7f3354171350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 -- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 msgr2=0x7f3354170e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 0x7f3354170e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 -- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 msgr2=0x7f3354051450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335bfff640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3354174070 con 0x7f33540e0170 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f335b7fe640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:11.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.365+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4017070 con 0x7f8b040c7910 2024-05-09T08:56:11.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940014070 con 0x7fe95006e480 2024-05-09T08:56:11.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af400c940 con 0x7f8b040c7910 2024-05-09T08:56:11.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af400cd10 con 0x7f8b040c7910 2024-05-09T08:56:11.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe940010890 con 0x7fe95006e480 2024-05-09T08:56:11.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f8b093db640 1 -- 172.21.15.49:0/2966518054 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0414e7c0 con 0x7f8b040c7910 2024-05-09T08:56:11.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f335bfff640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 0x7f3354171350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f335c068c10 tx=0x7f335c0692e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f8b093db640 1 -- 172.21.15.49:0/2966518054 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0414bea0 con 0x7f8b040c7910 2024-05-09T08:56:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0b8030 con 0x7f33540e0170 2024-05-09T08:56:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c0ba070 con 0x7f33540e0170 2024-05-09T08:56:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0be410 con 0x7f33540e0170 2024-05-09T08:56:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3354174330 con 0x7f33540e0170 2024-05-09T08:56:11.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940010c80 con 0x7fe95006e480 2024-05-09T08:56:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33541748f0 con 0x7f33540e0170 2024-05-09T08:56:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9501e4470 con 0x7fe95006e480 2024-05-09T08:56:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b3c01a070 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.366+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c04f5f0 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.367+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501e4a30 con 0x7fe95006e480 2024-05-09T08:56:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.368+0000 7f2b43fff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 0x7f2b2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.368+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0404f660 con 0x7f8b040c7910 2024-05-09T08:56:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.368+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2b3c089f10 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.368+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3354170750 con 0x7f33540e0170 2024-05-09T08:56:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95004f660 con 0x7fe95006e480 2024-05-09T08:56:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7f2b4affd640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 0x7f2b2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff638045070 con 0x7ff640071770 2024-05-09T08:56:11.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11ac096030 con 0x7f11a40dac60 2024-05-09T08:56:11.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7f11a8ff9640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 0x7f119006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7f2b4affd640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 0x7f2b2c06e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0cb280 tx=0x7f2b34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f11ac107a70 con 0x7f11a40dac60 2024-05-09T08:56:11.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.369+0000 7ff6277fe640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 0x7ff62c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7f11abfff640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 0x7f119006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6380ab8f0 con 0x7ff640071770 2024-05-09T08:56:11.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7ff63effd640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 0x7ff62c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f11ac068a60 con 0x7f11a40dac60 2024-05-09T08:56:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f335c0bc030 con 0x7f33540e0170 2024-05-09T08:56:11.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7f11abfff640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 0x7f119006e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11a0007b10 tx=0x7f11a0007a60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.370+0000 7ff63effd640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 0x7ff62c06e370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff63400a8b0 tx=0x7ff634008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.371+0000 7f3347fff640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 0x7f334006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.371+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f335c126af0 con 0x7f33540e0170 2024-05-09T08:56:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.371+0000 7f335b7fe640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 0x7f334006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.372+0000 7f335b7fe640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 0x7f334006e3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33500112c0 tx=0x7f3350009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.375+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff63807a310 con 0x7ff640071770 2024-05-09T08:56:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.376+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe940017070 con 0x7fe95006e480 2024-05-09T08:56:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.376+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8af401a070 con 0x7f8b040c7910 2024-05-09T08:56:11.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7fe93effd640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 0x7fe93006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7f8aeb7fe640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 0x7f8af006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b3c058300 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe94008afb0 con 0x7fe95006e480 2024-05-09T08:56:11.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8af408b5f0 con 0x7f8b040c7910 2024-05-09T08:56:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.377+0000 7fe94e59c640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 0x7fe93006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.378+0000 7f8b027fc640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 0x7f8af006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.378+0000 7fe94e59c640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 0x7fe93006e340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe944033000 tx=0x7fe944008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.378+0000 7f8b027fc640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 0x7f8af006e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b0414ceb0 tx=0x7f8afc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.379+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe9400b92d0 con 0x7fe95006e480 2024-05-09T08:56:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.385+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8af4059930 con 0x7f8b040c7910 2024-05-09T08:56:11.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.386+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f335c0f4ec0 con 0x7f33540e0170 2024-05-09T08:56:11.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.722+0000 7f93b2e19640 1 -- 172.21.15.49:0/940892369 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9394005580 con 0x7f93a40d5b30 2024-05-09T08:56:11.728 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:11.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.724+0000 7f93a27fc640 1 -- 172.21.15.49:0/940892369 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f939c059570 con 0x7f93a40d5b30 2024-05-09T08:56:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 msgr2=0x7f939806f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 0x7f939806f450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93a40df7b0 tx=0x7f93a8008040 comp rx=0 tx=0).stop 2024-05-09T08:56:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 msgr2=0x7f93a4052350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 0x7f93a4052350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f939c011960 tx=0x7f939c011e20 comp rx=0 tx=0).stop 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 shutdown_connections 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f939806cf90 0x7f939806f450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93a404c600 0x7f93a4195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93a404a9e0 0x7f93a4052890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 --2- 172.21.15.49:0/940892369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93a40d5b30 0x7f93a4052350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.734+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 >> 172.21.15.49:0/940892369 conn(0x7f93a400e6a0 msgr2=0x7f93a4009b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.735+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 shutdown_connections 2024-05-09T08:56:11.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.735+0000 7f938ffff640 1 -- 172.21.15.49:0/940892369 wait complete. 2024-05-09T08:56:11.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-05-09T08:56:11.752 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.786+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8b04166850 con 0x7f8b040c7910 2024-05-09T08:56:11.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.786+0000 7f8aeb7fe640 1 -- 172.21.15.49:0/2966518054 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8af405dc30 con 0x7f8b040c7910 2024-05-09T08:56:11.787 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.788+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 msgr2=0x7f8af006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.788+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 0x7f8af006e3c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b0414ceb0 tx=0x7f8afc00b040 comp rx=0 tx=0).stop 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.788+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 msgr2=0x7f8b04159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.788+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8af400e8c0 tx=0x7f8af400ed80 comp rx=0 tx=0).stop 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 shutdown_connections 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8af006bf00 0x7f8af006e3c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8b0414eaf0 0x7f8b04136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b040ccdb0 0x7f8b040cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 --2- 172.21.15.49:0/2966518054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8b040c7910 0x7f8b04159cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 >> 172.21.15.49:0/2966518054 conn(0x7f8b040cc290 msgr2=0x7f8b0413a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 shutdown_connections 2024-05-09T08:56:11.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f8ae97fa640 1 -- 172.21.15.49:0/2966518054 wait complete. 2024-05-09T08:56:11.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.789+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5ad8006e10 con 0x7f5ad80e66a0 2024-05-09T08:56:11.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.790+0000 7f5acbfff640 1 -- 172.21.15.49:0/3175175611 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ad4079d30 con 0x7f5ad80e66a0 2024-05-09T08:56:11.791 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 msgr2=0x7f5ac406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 0x7f5ac406e3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00a990 tx=0x7f5acc008040 comp rx=0 tx=0).stop 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 msgr2=0x7f5ad80460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 shutdown_connections 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5ac406bf00 0x7f5ac406e3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5ad8049500 0x7f5ad817a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ad80e66a0 0x7f5ad80460c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 --2- 172.21.15.49:0/3175175611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ad80dd470 0x7f5ad8048770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.793+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 >> 172.21.15.49:0/3175175611 conn(0x7f5ad8050a10 msgr2=0x7f5ad80dbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.794+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 shutdown_connections 2024-05-09T08:56:11.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.794+0000 7f5ae4a21640 1 -- 172.21.15.49:0/3175175611 wait complete. 2024-05-09T08:56:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.796+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f118c005850 con 0x7f11a40dac60 2024-05-09T08:56:11.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.796+0000 7f11a8ff9640 1 -- 172.21.15.49:0/654466039 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f11ac0d5e40 con 0x7f11a40dac60 2024-05-09T08:56:11.797 INFO:teuthology.orchestra.run.smithi049.stdout:55834574854 2024-05-09T08:56:11.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.799+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 msgr2=0x7f119006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.799+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 0x7f119006e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11a0007b10 tx=0x7f11a0007a60 comp rx=0 tx=0).stop 2024-05-09T08:56:11.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.799+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 msgr2=0x7f11a4184f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.799+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 0x7f11a4184f30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11ac050ef0 tx=0x7f11ac068930 comp rx=0 tx=0).stop 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 shutdown_connections 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f119006bf00 0x7f119006e3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a40dac60 0x7f11a4184f30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f11a4050ca0 0x7f11a40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 --2- 172.21.15.49:0/654466039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a404b500 0x7f11a4045340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 >> 172.21.15.49:0/654466039 conn(0x7f11a4050180 msgr2=0x7f11a40055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 shutdown_connections 2024-05-09T08:56:11.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.800+0000 7f11b13b0640 1 -- 172.21.15.49:0/654466039 wait complete. 2024-05-09T08:56:11.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-05-09T08:56:11.807 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-05-09T08:56:11.812 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-09T08:56:11.813 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.813+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2b4c154760 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.813+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3354005390 con 0x7f33540e0170 2024-05-09T08:56:11.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.813+0000 7f2b43fff640 1 -- 172.21.15.49:0/301596482 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b3c05c600 con 0x7f2b4c0ccbe0 2024-05-09T08:56:11.814 INFO:teuthology.orchestra.run.smithi049.stdout:55834574855 2024-05-09T08:56:11.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.814+0000 7f3347fff640 1 -- 172.21.15.49:0/616843902 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f335c0f91c0 con 0x7f33540e0170 2024-05-09T08:56:11.815 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-05-09T08:56:11.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.815+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff6400ccdb0 con 0x7ff640071770 2024-05-09T08:56:11.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.815+0000 7ff6277fe640 1 -- 172.21.15.49:0/4176065430 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff638079c10 con 0x7ff640071770 2024-05-09T08:56:11.816 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 msgr2=0x7f334006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 0x7f334006e3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33500112c0 tx=0x7f3350009190 comp rx=0 tx=0).stop 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 msgr2=0x7f3354171350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 0x7f3354171350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f335c068c10 tx=0x7f335c0692e0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 shutdown_connections 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f334006bf30 0x7f334006e3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f33540e0170 0x7f3354171350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33540d6e50 0x7f3354170e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 --2- 172.21.15.49:0/616843902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33540474e0 0x7f3354051450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 >> 172.21.15.49:0/616843902 conn(0x7f3354052390 msgr2=0x7f3354035b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 shutdown_connections 2024-05-09T08:56:11.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.816+0000 7f336098c640 1 -- 172.21.15.49:0/616843902 wait complete. 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 msgr2=0x7f2b2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 0x7f2b2c06e320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0cb280 tx=0x7f2b34008040 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 msgr2=0x7f2b4c148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 0x7f2b4c148a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00c800 tx=0x7f2b3c00ccc0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 shutdown_connections 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2b2c06be60 0x7f2b2c06e320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b4c157790 0x7f2b4c1461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b4c0ccbe0 0x7f2b4c148a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 --2- 172.21.15.49:0/301596482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b4c06e140 0x7f2b4c147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 >> 172.21.15.49:0/301596482 conn(0x7f2b4c0cc0f0 msgr2=0x7f2b4c146e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 shutdown_connections 2024-05-09T08:56:11.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.817+0000 7f2b4bfff640 1 -- 172.21.15.49:0/301596482 wait complete. 2024-05-09T08:56:11.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 msgr2=0x7ff62c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 0x7ff62c06e370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff63400a8b0 tx=0x7ff634008040 comp rx=0 tx=0).stop 2024-05-09T08:56:11.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 msgr2=0x7ff64012c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff63802a2e0 tx=0x7ff6380046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 shutdown_connections 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff62c06beb0 0x7ff62c06e370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6400c3450 0x7ff64012c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff640071770 0x7ff64012c2b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 --2- 172.21.15.49:0/4176065430 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff640068560 0x7ff6400c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 >> 172.21.15.49:0/4176065430 conn(0x7ff640074f20 msgr2=0x7ff64014f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 shutdown_connections 2024-05-09T08:56:11.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.818+0000 7ff63ffff640 1 -- 172.21.15.49:0/4176065430 wait complete. 2024-05-09T08:56:11.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.823+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe950071270 con 0x7fe95006e480 2024-05-09T08:56:11.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.824+0000 7fe93effd640 1 -- 172.21.15.49:0/2903882327 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe940010e20 con 0x7fe95006e480 2024-05-09T08:56:11.824 INFO:teuthology.orchestra.run.smithi049.stdout:55834574853 2024-05-09T08:56:11.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 msgr2=0x7fe93006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 0x7fe93006e340 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe944033000 tx=0x7fe944008030 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 msgr2=0x7fe95014f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b480 tx=0x7fe94000b940 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 shutdown_connections 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe93006be80 0x7fe93006e340 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe950155060 0x7fe95014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe95006e480 0x7fe95014f490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 --2- 172.21.15.49:0/2903882327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95006bdd0 0x7fe9501e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 >> 172.21.15.49:0/2903882327 conn(0x7fe9500cc290 msgr2=0x7fe95006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 shutdown_connections 2024-05-09T08:56:11.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.826+0000 7fe954d32640 1 -- 172.21.15.49:0/2903882327 wait complete. 2024-05-09T08:56:11.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-05-09T08:56:11.830 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-05-09T08:56:11.836 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-05-09T08:56:11.837 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-05-09T08:56:11.838 DEBUG:teuthology.parallel:result is None 2024-05-09T08:56:11.839 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T08:56:11.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:11.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.981+0000 7f90b0cf7640 1 Processor -- start 2024-05-09T08:56:11.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.981+0000 7f90b0cf7640 1 -- start start 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90ac0c66c0 0x7f90ac1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f90ac0c6e00 0x7f90ac06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f90ac0c3900 con 0x7f90ac0c66c0 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f90ac158f20 con 0x7f90ac0c6e00 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90b0cf7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac1598f0 con 0x7f90ac072300 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90ac0c66c0 0x7f90ac1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f90ac0c66c0 0x7f90ac1593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48360/0 (socket says 172.21.15.49:48360) 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 -- 172.21.15.49:0/3732038654 learned_addr learned my addr 172.21.15.49:0/3732038654 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90aad9d640 1 --2- 172.21.15.49:0/3732038654 >> v2:172.21.15.183:3300/0 conn(0x7f90ac0c6e00 0x7f90ac06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90aa59c640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 -- 172.21.15.49:0/3732038654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 msgr2=0x7f90ac0726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 -- 172.21.15.49:0/3732038654 >> v2:172.21.15.183:3300/0 conn(0x7f90ac0c6e00 msgr2=0x7f90ac06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:11.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 --2- 172.21.15.49:0/3732038654 >> v2:172.21.15.183:3300/0 conn(0x7f90ac0c6e00 0x7f90ac06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.982+0000 7f90a3fff640 1 -- 172.21.15.49:0/3732038654 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac06e980 con 0x7f90ac0c66c0 2024-05-09T08:56:11.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.983+0000 7f90a3fff640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac1593b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0cd4a0 tx=0x7f909402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb40f71c3f7a45a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.983+0000 7f90a9d9b640 1 -- 172.21.15.49:0/3732038654 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094037070 con 0x7f90ac0c66c0 2024-05-09T08:56:11.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.983+0000 7f90a9d9b640 1 -- 172.21.15.49:0/3732038654 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9094032440 con 0x7f90ac0c66c0 2024-05-09T08:56:11.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.983+0000 7f90a9d9b640 1 -- 172.21.15.49:0/3732038654 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094032810 con 0x7f90ac0c66c0 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 -- 172.21.15.49:0/3732038654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 msgr2=0x7f90ac1593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac1593b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0cd4a0 tx=0x7f909402a1e0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 -- 172.21.15.49:0/3732038654 shutdown_connections 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/3732038654 >> v2:172.21.15.183:3300/0 conn(0x7f90ac0c6e00 0x7f90ac06e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac1593b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/3732038654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 -- 172.21.15.49:0/3732038654 >> 172.21.15.49:0/3732038654 conn(0x7f90ac074840 msgr2=0x7f90ac074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 -- 172.21.15.49:0/3732038654 shutdown_connections 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.984+0000 7f90b0cf7640 1 -- 172.21.15.49:0/3732038654 wait complete. 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.985+0000 7f90b0cf7640 1 Processor -- start 2024-05-09T08:56:11.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.985+0000 7f90b0cf7640 1 -- start start 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90ac0c6e00 0x7f90ac1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0c3e30 con 0x7f90ac0c66c0 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0c3b50 con 0x7f90ac0c6e00 2024-05-09T08:56:11.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90b0cf7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac158bb0 con 0x7f90ac072300 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48364/0 (socket says 172.21.15.49:48364) 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 -- 172.21.15.49:0/715481779 learned_addr learned my addr 172.21.15.49:0/715481779 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90aad9d640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90ac0c6e00 0x7f90ac1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90aa59c640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 -- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 msgr2=0x7f90ac0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 -- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90ac0c6e00 msgr2=0x7f90ac1513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90ac0c6e00 0x7f90ac1513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.986+0000 7f90a3fff640 1 -- 172.21.15.49:0/715481779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac15c7e0 con 0x7f90ac0c66c0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90a3fff640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f90940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094041070 con 0x7f90ac0c66c0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9094032e50 con 0x7f90ac0c66c0 2024-05-09T08:56:11.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac15b980 con 0x7f90ac0c66c0 2024-05-09T08:56:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094037040 con 0x7f90ac0c66c0 2024-05-09T08:56:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.987+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac15bf40 con 0x7f90ac0c66c0 2024-05-09T08:56:11.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.988+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac14f9e0 con 0x7f90ac0c66c0 2024-05-09T08:56:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.988+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9094044070 con 0x7f90ac0c66c0 2024-05-09T08:56:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.989+0000 7f90a2ffd640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 0x7f908c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.989+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90940ab740 con 0x7f90ac0c66c0 2024-05-09T08:56:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.989+0000 7f90aa59c640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 0x7f908c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:11.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.989+0000 7f90aa59c640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 0x7f908c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f909c009b40 tx=0x7f909c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:11.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:11.993+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f90ac14f9e0 con 0x7f90ac0c66c0 2024-05-09T08:56:12.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.173+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90ac0c34c0 con 0x7f908c06be60 2024-05-09T08:56:12.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.179+0000 7f90a2ffd640 1 -- 172.21.15.49:0/715481779 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f90ac0c34c0 con 0x7f908c06be60 2024-05-09T08:56:12.182 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:12.183 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 msgr2=0x7f908c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 0x7f908c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f909c009b40 tx=0x7f909c009190 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 msgr2=0x7f90ac150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9094000c00 tx=0x7f90940046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 shutdown_connections 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f908c06be60 0x7f908c06e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90ac0c6e00 0x7f90ac1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f90ac0c66c0 0x7f90ac150eb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 --2- 172.21.15.49:0/715481779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f90ac072300 0x7f90ac0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.182+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 >> 172.21.15.49:0/715481779 conn(0x7f90ac074840 msgr2=0x7f90ac14d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.183+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 shutdown_connections 2024-05-09T08:56:12.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.183+0000 7f90b0cf7640 1 -- 172.21.15.49:0/715481779 wait complete. 2024-05-09T08:56:12.194 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-09T08:56:10.573324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235592,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739128,"statfs":{"total":773094113280,"available":772852867072,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.897114"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651856+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:03.651856+0000","last_peered":"2024-05-09T08:56:03.651856+0000","last_clean":"2024-05-09T08:56:03.651856+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:03.651856+0000","last_undegraded":"2024-05-09T08:56:03.651856+0000","last_fullsized":"2024-05-09T08:56:03.651856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005327+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:03.005327+0000","last_peered":"2024-05-09T08:56:03.005327+0000","last_clean":"2024-05-09T08:56:03.005327+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:03.005327+0000","last_undegraded":"2024-05-09T08:56:03.005327+0000","last_fullsized":"2024-05-09T08:56:03.005327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651547+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:03.651547+0000","last_peered":"2024-05-09T08:56:03.651547+0000","last_clean":"2024-05-09T08:56:03.651547+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:03.651547+0000","last_undegraded":"2024-05-09T08:56:03.651547+0000","last_fullsized":"2024-05-09T08:56:03.651547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077137+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:03.077137+0000","last_peered":"2024-05-09T08:56:03.077137+0000","last_clean":"2024-05-09T08:56:03.077137+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:03.077137+0000","last_undegraded":"2024-05-09T08:56:03.077137+0000","last_fullsized":"2024-05-09T08:56:03.077137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005453+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:03.005453+0000","last_peered":"2024-05-09T08:56:03.005453+0000","last_clean":"2024-05-09T08:56:03.005453+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:03.005453+0000","last_undegraded":"2024-05-09T08:56:03.005453+0000","last_fullsized":"2024-05-09T08:56:03.005453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077401+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:03.077401+0000","last_peered":"2024-05-09T08:56:03.077401+0000","last_clean":"2024-05-09T08:56:03.077401+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:03.077401+0000","last_undegraded":"2024-05-09T08:56:03.077401+0000","last_fullsized":"2024-05-09T08:56:03.077401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686992+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:06.686992+0000","last_peered":"2024-05-09T08:56:06.686992+0000","last_clean":"2024-05-09T08:56:06.686992+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:06.686992+0000","last_undegraded":"2024-05-09T08:56:06.686992+0000","last_fullsized":"2024-05-09T08:56:06.686992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686097+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:06.686097+0000","last_peered":"2024-05-09T08:56:06.686097+0000","last_clean":"2024-05-09T08:56:06.686097+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:06.686097+0000","last_undegraded":"2024-05-09T08:56:06.686097+0000","last_fullsized":"2024-05-09T08:56:06.686097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684898+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:06.684898+0000","last_peered":"2024-05-09T08:56:06.684898+0000","last_clean":"2024-05-09T08:56:06.684898+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:06.684898+0000","last_undegraded":"2024-05-09T08:56:06.684898+0000","last_fullsized":"2024-05-09T08:56:06.684898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077321+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:03.077321+0000","last_peered":"2024-05-09T08:56:03.077321+0000","last_clean":"2024-05-09T08:56:03.077321+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:03.077321+0000","last_undegraded":"2024-05-09T08:56:03.077321+0000","last_fullsized":"2024-05-09T08:56:03.077321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005220+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:03.005220+0000","last_peered":"2024-05-09T08:56:03.005220+0000","last_clean":"2024-05-09T08:56:03.005220+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:03.005220+0000","last_undegraded":"2024-05-09T08:56:03.005220+0000","last_fullsized":"2024-05-09T08:56:03.005220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.687706+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:06.687706+0000","last_peered":"2024-05-09T08:56:06.687706+0000","last_clean":"2024-05-09T08:56:06.687706+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:06.687706+0000","last_undegraded":"2024-05-09T08:56:06.687706+0000","last_fullsized":"2024-05-09T08:56:06.687706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695615+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:02.695615+0000","last_peered":"2024-05-09T08:56:02.695615+0000","last_clean":"2024-05-09T08:56:02.695615+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:02.695615+0000","last_undegraded":"2024-05-09T08:56:02.695615+0000","last_fullsized":"2024-05-09T08:56:02.695615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685615+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:06.685615+0000","last_peered":"2024-05-09T08:56:06.685615+0000","last_clean":"2024-05-09T08:56:06.685615+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:06.685615+0000","last_undegraded":"2024-05-09T08:56:06.685615+0000","last_fullsized":"2024-05-09T08:56:06.685615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693521+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:02.693521+0000","last_peered":"2024-05-09T08:56:02.693521+0000","last_clean":"2024-05-09T08:56:02.693521+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:02.693521+0000","last_undegraded":"2024-05-09T08:56:02.693521+0000","last_fullsized":"2024-05-09T08:56:02.693521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688423+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:02.688423+0000","last_peered":"2024-05-09T08:56:02.688423+0000","last_clean":"2024-05-09T08:56:02.688423+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:02.688423+0000","last_undegraded":"2024-05-09T08:56:02.688423+0000","last_fullsized":"2024-05-09T08:56:02.688423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077019+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:03.077019+0000","last_peered":"2024-05-09T08:56:03.077019+0000","last_clean":"2024-05-09T08:56:03.077019+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:03.077019+0000","last_undegraded":"2024-05-09T08:56:03.077019+0000","last_fullsized":"2024-05-09T08:56:03.077019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651254+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:03.651254+0000","last_peered":"2024-05-09T08:56:03.651254+0000","last_clean":"2024-05-09T08:56:03.651254+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:03.651254+0000","last_undegraded":"2024-05-09T08:56:03.651254+0000","last_fullsized":"2024-05-09T08:56:03.651254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688243+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:02.688243+0000","last_peered":"2024-05-09T08:56:02.688243+0000","last_clean":"2024-05-09T08:56:02.688243+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:02.688243+0000","last_undegraded":"2024-05-09T08:56:02.688243+0000","last_fullsized":"2024-05-09T08:56:02.688243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077432+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:03.077432+0000","last_peered":"2024-05-09T08:56:03.077432+0000","last_clean":"2024-05-09T08:56:03.077432+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:03.077432+0000","last_undegraded":"2024-05-09T08:56:03.077432+0000","last_fullsized":"2024-05-09T08:56:03.077432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693243+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693243+0000","last_peered":"2024-05-09T08:56:02.693243+0000","last_clean":"2024-05-09T08:56:02.693243+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693243+0000","last_undegraded":"2024-05-09T08:56:02.693243+0000","last_fullsized":"2024-05-09T08:56:02.693243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651913+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:03.651913+0000","last_peered":"2024-05-09T08:56:03.651913+0000","last_clean":"2024-05-09T08:56:03.651913+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:03.651913+0000","last_undegraded":"2024-05-09T08:56:03.651913+0000","last_fullsized":"2024-05-09T08:56:03.651913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693454+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:02.693454+0000","last_peered":"2024-05-09T08:56:02.693454+0000","last_clean":"2024-05-09T08:56:02.693454+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:02.693454+0000","last_undegraded":"2024-05-09T08:56:02.693454+0000","last_fullsized":"2024-05-09T08:56:02.693454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685261+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:06.685261+0000","last_peered":"2024-05-09T08:56:06.685261+0000","last_clean":"2024-05-09T08:56:06.685261+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:06.685261+0000","last_undegraded":"2024-05-09T08:56:06.685261+0000","last_fullsized":"2024-05-09T08:56:06.685261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651234+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:03.651234+0000","last_peered":"2024-05-09T08:56:03.651234+0000","last_clean":"2024-05-09T08:56:03.651234+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:03.651234+0000","last_undegraded":"2024-05-09T08:56:03.651234+0000","last_fullsized":"2024-05-09T08:56:03.651234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693333+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:02.693333+0000","last_peered":"2024-05-09T08:56:02.693333+0000","last_clean":"2024-05-09T08:56:02.693333+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:02.693333+0000","last_undegraded":"2024-05-09T08:56:02.693333+0000","last_fullsized":"2024-05-09T08:56:02.693333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693327+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:02.693327+0000","last_peered":"2024-05-09T08:56:02.693327+0000","last_clean":"2024-05-09T08:56:02.693327+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:02.693327+0000","last_undegraded":"2024-05-09T08:56:02.693327+0000","last_fullsized":"2024-05-09T08:56:02.693327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686743+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:06.686743+0000","last_peered":"2024-05-09T08:56:06.686743+0000","last_clean":"2024-05-09T08:56:06.686743+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:06.686743+0000","last_undegraded":"2024-05-09T08:56:06.686743+0000","last_fullsized":"2024-05-09T08:56:06.686743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684661+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:06.684661+0000","last_peered":"2024-05-09T08:56:06.684661+0000","last_clean":"2024-05-09T08:56:06.684661+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:06.684661+0000","last_undegraded":"2024-05-09T08:56:06.684661+0000","last_fullsized":"2024-05-09T08:56:06.684661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684638+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:06.684638+0000","last_peered":"2024-05-09T08:56:06.684638+0000","last_clean":"2024-05-09T08:56:06.684638+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:06.684638+0000","last_undegraded":"2024-05-09T08:56:06.684638+0000","last_fullsized":"2024-05-09T08:56:06.684638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652037+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:03.652037+0000","last_peered":"2024-05-09T08:56:03.652037+0000","last_clean":"2024-05-09T08:56:03.652037+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:03.652037+0000","last_undegraded":"2024-05-09T08:56:03.652037+0000","last_fullsized":"2024-05-09T08:56:03.652037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686308+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:06.686308+0000","last_peered":"2024-05-09T08:56:06.686308+0000","last_clean":"2024-05-09T08:56:06.686308+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:06.686308+0000","last_undegraded":"2024-05-09T08:56:06.686308+0000","last_fullsized":"2024-05-09T08:56:06.686308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:12.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:12.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.337+0000 7f1e4a8c1640 1 Processor -- start 2024-05-09T08:56:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.337+0000 7f1e4a8c1640 1 -- start start 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e4a8c1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e44154f60 0x7f1e4406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e4a8c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e44155340 0x7f1e4406b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e4a8c1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e4415b390 con 0x7f1e440729e0 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e4a8c1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1e4415c2d0 con 0x7f1e44154f60 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e4a8c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4415b670 con 0x7f1e44155340 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e44154f60 0x7f1e4406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e44154f60 0x7f1e4406b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48380/0 (socket says 172.21.15.49:48380) 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 -- 172.21.15.49:0/1259933935 learned_addr learned my addr 172.21.15.49:0/1259933935 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 -- 172.21.15.49:0/1259933935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e44155340 msgr2=0x7f1e4406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 --2- 172.21.15.49:0/1259933935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e44155340 0x7f1e4406b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 -- 172.21.15.49:0/1259933935 >> v1:172.21.15.183:6789/0 conn(0x7f1e440729e0 legacy=0x7f1e4415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.338+0000 7f1e43fff640 1 -- 172.21.15.49:0/1259933935 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4415b950 con 0x7f1e44154f60 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.339+0000 7f1e43fff640 1 --2- 172.21.15.49:0/1259933935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e44154f60 0x7f1e4406b200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009870 tx=0x7f1e3402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f4aa920204cb02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.339+0000 7f1e437fe640 1 -- 172.21.15.49:0/1259933935 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34037070 con 0x7f1e44154f60 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.339+0000 7f1e437fe640 1 -- 172.21.15.49:0/1259933935 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e34032440 con 0x7f1e44154f60 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.339+0000 7f1e437fe640 1 -- 172.21.15.49:0/1259933935 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34032810 con 0x7f1e44154f60 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/1259933935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e44154f60 msgr2=0x7f1e4406b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/1259933935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e44154f60 0x7f1e4406b200 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009870 tx=0x7f1e3402a2a0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/1259933935 shutdown_connections 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/1259933935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e44155340 0x7f1e4406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/1259933935 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e44154f60 0x7f1e4406b200 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/1259933935 >> 172.21.15.49:0/1259933935 conn(0x7f1e44067d90 msgr2=0x7f1e44074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/1259933935 shutdown_connections 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.340+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/1259933935 wait complete. 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.341+0000 7f1e4a8c1640 1 Processor -- start 2024-05-09T08:56:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.341+0000 7f1e4a8c1640 1 -- start start 2024-05-09T08:56:12.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e440729e0 0x7f1e4415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e44154f60 0x7f1e440c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44073210 con 0x7f1e440c8550 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4415c520 con 0x7f1e44154f60 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e4406d110 con 0x7f1e440729e0 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48394/0 (socket says 172.21.15.49:48394) 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 -- 172.21.15.49:0/690752163 learned_addr learned my addr 172.21.15.49:0/690752163 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e3bfff640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e44154f60 0x7f1e440c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e43fff640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e440729e0 0x7f1e4415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 -- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e440729e0 msgr2=0x7f1e4415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e440729e0 0x7f1e4415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 -- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e44154f60 msgr2=0x7f1e440c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e44154f60 0x7f1e440c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 -- 172.21.15.49:0/690752163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e440ccbe0 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e48e5e640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c7b0 tx=0x7f1e3c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c017070 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3c003da0 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.342+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e441f1570 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.343+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0052b0 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.343+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e441572c0 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.343+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e44068560 con 0x7f1e440c8550 2024-05-09T08:56:12.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.344+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e3c01a070 con 0x7f1e440c8550 2024-05-09T08:56:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.344+0000 7f1e417fa640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 0x7f1e2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.345+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1e3c08a450 con 0x7f1e440c8550 2024-05-09T08:56:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.345+0000 7f1e43fff640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 0x7f1e2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.345+0000 7f1e43fff640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 0x7f1e2406e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e34034000 tx=0x7f1e3402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.350+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e3c0587b0 con 0x7f1e440c8550 2024-05-09T08:56:12.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.528+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e440cc370 con 0x7f1e2406be60 2024-05-09T08:56:12.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.536+0000 7f1e417fa640 1 -- 172.21.15.49:0/690752163 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f1e440cc370 con 0x7f1e2406be60 2024-05-09T08:56:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:12.539 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 msgr2=0x7f1e2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 0x7f1e2406e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e34034000 tx=0x7f1e3402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 msgr2=0x7f1e440c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c7b0 tx=0x7f1e3c00cc70 comp rx=0 tx=0).stop 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 shutdown_connections 2024-05-09T08:56:12.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2406be60 0x7f1e2406e320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e440c8550 0x7f1e440c6230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e44154f60 0x7f1e440c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 --2- 172.21.15.49:0/690752163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e440729e0 0x7f1e4415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 >> 172.21.15.49:0/690752163 conn(0x7f1e44067d90 msgr2=0x7f1e44159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 shutdown_connections 2024-05-09T08:56:12.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.538+0000 7f1e4a8c1640 1 -- 172.21.15.49:0/690752163 wait complete. 2024-05-09T08:56:12.549 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-09T08:56:10.573324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235592,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739128,"statfs":{"total":773094113280,"available":772852867072,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.897114"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651856+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:03.651856+0000","last_peered":"2024-05-09T08:56:03.651856+0000","last_clean":"2024-05-09T08:56:03.651856+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:03.651856+0000","last_undegraded":"2024-05-09T08:56:03.651856+0000","last_fullsized":"2024-05-09T08:56:03.651856+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005327+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:03.005327+0000","last_peered":"2024-05-09T08:56:03.005327+0000","last_clean":"2024-05-09T08:56:03.005327+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:03.005327+0000","last_undegraded":"2024-05-09T08:56:03.005327+0000","last_fullsized":"2024-05-09T08:56:03.005327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651547+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:03.651547+0000","last_peered":"2024-05-09T08:56:03.651547+0000","last_clean":"2024-05-09T08:56:03.651547+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:03.651547+0000","last_undegraded":"2024-05-09T08:56:03.651547+0000","last_fullsized":"2024-05-09T08:56:03.651547+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077137+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:03.077137+0000","last_peered":"2024-05-09T08:56:03.077137+0000","last_clean":"2024-05-09T08:56:03.077137+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:03.077137+0000","last_undegraded":"2024-05-09T08:56:03.077137+0000","last_fullsized":"2024-05-09T08:56:03.077137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005453+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:03.005453+0000","last_peered":"2024-05-09T08:56:03.005453+0000","last_clean":"2024-05-09T08:56:03.005453+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:03.005453+0000","last_undegraded":"2024-05-09T08:56:03.005453+0000","last_fullsized":"2024-05-09T08:56:03.005453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077401+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:03.077401+0000","last_peered":"2024-05-09T08:56:03.077401+0000","last_clean":"2024-05-09T08:56:03.077401+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:03.077401+0000","last_undegraded":"2024-05-09T08:56:03.077401+0000","last_fullsized":"2024-05-09T08:56:03.077401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686992+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:06.686992+0000","last_peered":"2024-05-09T08:56:06.686992+0000","last_clean":"2024-05-09T08:56:06.686992+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:06.686992+0000","last_undegraded":"2024-05-09T08:56:06.686992+0000","last_fullsized":"2024-05-09T08:56:06.686992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686097+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:06.686097+0000","last_peered":"2024-05-09T08:56:06.686097+0000","last_clean":"2024-05-09T08:56:06.686097+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:06.686097+0000","last_undegraded":"2024-05-09T08:56:06.686097+0000","last_fullsized":"2024-05-09T08:56:06.686097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684898+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:06.684898+0000","last_peered":"2024-05-09T08:56:06.684898+0000","last_clean":"2024-05-09T08:56:06.684898+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:06.684898+0000","last_undegraded":"2024-05-09T08:56:06.684898+0000","last_fullsized":"2024-05-09T08:56:06.684898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077321+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:03.077321+0000","last_peered":"2024-05-09T08:56:03.077321+0000","last_clean":"2024-05-09T08:56:03.077321+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:03.077321+0000","last_undegraded":"2024-05-09T08:56:03.077321+0000","last_fullsized":"2024-05-09T08:56:03.077321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005220+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:03.005220+0000","last_peered":"2024-05-09T08:56:03.005220+0000","last_clean":"2024-05-09T08:56:03.005220+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:03.005220+0000","last_undegraded":"2024-05-09T08:56:03.005220+0000","last_fullsized":"2024-05-09T08:56:03.005220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.687706+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:06.687706+0000","last_peered":"2024-05-09T08:56:06.687706+0000","last_clean":"2024-05-09T08:56:06.687706+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:06.687706+0000","last_undegraded":"2024-05-09T08:56:06.687706+0000","last_fullsized":"2024-05-09T08:56:06.687706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695615+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:02.695615+0000","last_peered":"2024-05-09T08:56:02.695615+0000","last_clean":"2024-05-09T08:56:02.695615+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:02.695615+0000","last_undegraded":"2024-05-09T08:56:02.695615+0000","last_fullsized":"2024-05-09T08:56:02.695615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685615+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:06.685615+0000","last_peered":"2024-05-09T08:56:06.685615+0000","last_clean":"2024-05-09T08:56:06.685615+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:06.685615+0000","last_undegraded":"2024-05-09T08:56:06.685615+0000","last_fullsized":"2024-05-09T08:56:06.685615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693521+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:02.693521+0000","last_peered":"2024-05-09T08:56:02.693521+0000","last_clean":"2024-05-09T08:56:02.693521+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:02.693521+0000","last_undegraded":"2024-05-09T08:56:02.693521+0000","last_fullsized":"2024-05-09T08:56:02.693521+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688423+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:02.688423+0000","last_peered":"2024-05-09T08:56:02.688423+0000","last_clean":"2024-05-09T08:56:02.688423+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:02.688423+0000","last_undegraded":"2024-05-09T08:56:02.688423+0000","last_fullsized":"2024-05-09T08:56:02.688423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077019+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:03.077019+0000","last_peered":"2024-05-09T08:56:03.077019+0000","last_clean":"2024-05-09T08:56:03.077019+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:03.077019+0000","last_undegraded":"2024-05-09T08:56:03.077019+0000","last_fullsized":"2024-05-09T08:56:03.077019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651254+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:03.651254+0000","last_peered":"2024-05-09T08:56:03.651254+0000","last_clean":"2024-05-09T08:56:03.651254+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:03.651254+0000","last_undegraded":"2024-05-09T08:56:03.651254+0000","last_fullsized":"2024-05-09T08:56:03.651254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688243+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:02.688243+0000","last_peered":"2024-05-09T08:56:02.688243+0000","last_clean":"2024-05-09T08:56:02.688243+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:02.688243+0000","last_undegraded":"2024-05-09T08:56:02.688243+0000","last_fullsized":"2024-05-09T08:56:02.688243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077432+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:03.077432+0000","last_peered":"2024-05-09T08:56:03.077432+0000","last_clean":"2024-05-09T08:56:03.077432+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:03.077432+0000","last_undegraded":"2024-05-09T08:56:03.077432+0000","last_fullsized":"2024-05-09T08:56:03.077432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693243+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693243+0000","last_peered":"2024-05-09T08:56:02.693243+0000","last_clean":"2024-05-09T08:56:02.693243+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693243+0000","last_undegraded":"2024-05-09T08:56:02.693243+0000","last_fullsized":"2024-05-09T08:56:02.693243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651913+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:03.651913+0000","last_peered":"2024-05-09T08:56:03.651913+0000","last_clean":"2024-05-09T08:56:03.651913+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:03.651913+0000","last_undegraded":"2024-05-09T08:56:03.651913+0000","last_fullsized":"2024-05-09T08:56:03.651913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693454+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:02.693454+0000","last_peered":"2024-05-09T08:56:02.693454+0000","last_clean":"2024-05-09T08:56:02.693454+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:02.693454+0000","last_undegraded":"2024-05-09T08:56:02.693454+0000","last_fullsized":"2024-05-09T08:56:02.693454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685261+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:06.685261+0000","last_peered":"2024-05-09T08:56:06.685261+0000","last_clean":"2024-05-09T08:56:06.685261+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:06.685261+0000","last_undegraded":"2024-05-09T08:56:06.685261+0000","last_fullsized":"2024-05-09T08:56:06.685261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651234+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:03.651234+0000","last_peered":"2024-05-09T08:56:03.651234+0000","last_clean":"2024-05-09T08:56:03.651234+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:03.651234+0000","last_undegraded":"2024-05-09T08:56:03.651234+0000","last_fullsized":"2024-05-09T08:56:03.651234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693333+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:02.693333+0000","last_peered":"2024-05-09T08:56:02.693333+0000","last_clean":"2024-05-09T08:56:02.693333+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:02.693333+0000","last_undegraded":"2024-05-09T08:56:02.693333+0000","last_fullsized":"2024-05-09T08:56:02.693333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693327+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:02.693327+0000","last_peered":"2024-05-09T08:56:02.693327+0000","last_clean":"2024-05-09T08:56:02.693327+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:02.693327+0000","last_undegraded":"2024-05-09T08:56:02.693327+0000","last_fullsized":"2024-05-09T08:56:02.693327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686743+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:06.686743+0000","last_peered":"2024-05-09T08:56:06.686743+0000","last_clean":"2024-05-09T08:56:06.686743+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:06.686743+0000","last_undegraded":"2024-05-09T08:56:06.686743+0000","last_fullsized":"2024-05-09T08:56:06.686743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684661+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:06.684661+0000","last_peered":"2024-05-09T08:56:06.684661+0000","last_clean":"2024-05-09T08:56:06.684661+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:06.684661+0000","last_undegraded":"2024-05-09T08:56:06.684661+0000","last_fullsized":"2024-05-09T08:56:06.684661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.684638+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:06.684638+0000","last_peered":"2024-05-09T08:56:06.684638+0000","last_clean":"2024-05-09T08:56:06.684638+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:06.684638+0000","last_undegraded":"2024-05-09T08:56:06.684638+0000","last_fullsized":"2024-05-09T08:56:06.684638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652037+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:03.652037+0000","last_peered":"2024-05-09T08:56:03.652037+0000","last_clean":"2024-05-09T08:56:03.652037+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:03.652037+0000","last_undegraded":"2024-05-09T08:56:03.652037+0000","last_fullsized":"2024-05-09T08:56:03.652037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686308+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:06.686308+0000","last_peered":"2024-05-09T08:56:06.686308+0000","last_clean":"2024-05-09T08:56:06.686308+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:06.686308+0000","last_undegraded":"2024-05-09T08:56:06.686308+0000","last_fullsized":"2024-05-09T08:56:06.686308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:12.553 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T08:56:12.553 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T08:56:12.553 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T08:56:12.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:56:12.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.691+0000 7f4bd77d5640 1 Processor -- start 2024-05-09T08:56:12.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.691+0000 7f4bd77d5640 1 -- start start 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd77d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bd0154f40 0x7f4bd015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd77d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd015eef0 0x7f4bd00cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd77d5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd015b1b0 con 0x7f4bd00cb290 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd77d5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd00b3fd0 con 0x7f4bd0154f40 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd77d5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd015bd30 con 0x7f4bd015eef0 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd5571640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bd0154f40 0x7f4bd015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.693 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd5571640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4bd0154f40 0x7f4bd015e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46858/0 (socket says 172.21.15.49:46858) 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.692+0000 7f4bd5571640 1 -- 172.21.15.49:0/2882552915 learned_addr learned my addr 172.21.15.49:0/2882552915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5d72640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd015eef0 0x7f4bd00cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5571640 1 -- 172.21.15.49:0/2882552915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd015eef0 msgr2=0x7f4bd00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5571640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd015eef0 0x7f4bd00cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5571640 1 -- 172.21.15.49:0/2882552915 >> v1:172.21.15.183:6789/0 conn(0x7f4bd00cb290 legacy=0x7f4bd00cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5571640 1 -- 172.21.15.49:0/2882552915 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd015b490 con 0x7f4bd0154f40 2024-05-09T08:56:12.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.693+0000 7f4bd5571640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd015e9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009870 tx=0x7f4bc802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6095de96a8fb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bcffff640 1 -- 172.21.15.49:0/2882552915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8037070 con 0x7f4bd0154f40 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bcffff640 1 -- 172.21.15.49:0/2882552915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc8032440 con 0x7f4bd0154f40 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bcffff640 1 -- 172.21.15.49:0/2882552915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8032810 con 0x7f4bd0154f40 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2882552915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 msgr2=0x7f4bd015e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd015e9b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009870 tx=0x7f4bc802a1c0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2882552915 shutdown_connections 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd015eef0 0x7f4bd00cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2882552915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd015e9b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2882552915 >> 172.21.15.49:0/2882552915 conn(0x7f4bd0067b80 msgr2=0x7f4bd0074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2882552915 shutdown_connections 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.694+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2882552915 wait complete. 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.695+0000 7f4bd77d5640 1 Processor -- start 2024-05-09T08:56:12.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.695+0000 7f4bd77d5640 1 -- start start 2024-05-09T08:56:12.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd00cb290 0x7f4bd01f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd015eef0 0x7f4bd00cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd015e490 con 0x7f4bd0154f40 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd015ba50 con 0x7f4bd015eef0 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd77d5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd00b3580 con 0x7f4bd00cb290 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd4d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd4d70640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46862/0 (socket says 172.21.15.49:46862) 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd4d70640 1 -- 172.21.15.49:0/2635900644 learned_addr learned my addr 172.21.15.49:0/2635900644 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd5d72640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd015eef0 0x7f4bd00cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.696+0000 7f4bd5571640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd00cb290 0x7f4bd01f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 -- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd00cb290 msgr2=0x7f4bd01f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd00cb290 0x7f4bd01f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 -- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd015eef0 msgr2=0x7f4bd00cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd015eef0 0x7f4bd00cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 -- 172.21.15.49:0/2635900644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00ca0c0 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd4d70640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c430 tx=0x7f4bc400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4017070 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc40052d0 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd00ca8a0 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4003830 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.697+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01f1490 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.698+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd00d1310 con 0x7f4bd0154f40 2024-05-09T08:56:12.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.698+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bc401a070 con 0x7f4bd0154f40 2024-05-09T08:56:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.699+0000 7f4bcdffb640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 0x7f4bbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.699+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4bc408a1c0 con 0x7f4bd0154f40 2024-05-09T08:56:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.699+0000 7f4bd5571640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 0x7f4bbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:12.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.699+0000 7f4bd5571640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 0x7f4bbc06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8034000 tx=0x7f4bc802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.704+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4bd00d1310 con 0x7f4bd0154f40 2024-05-09T08:56:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.929+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4bd0068560 con 0x7f4bd0154f40 2024-05-09T08:56:12.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.929+0000 7f4bcdffb640 1 -- 172.21.15.49:0/2635900644 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f4bc4058490 con 0x7f4bd0154f40 2024-05-09T08:56:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:12.929 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 msgr2=0x7f4bbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 0x7f4bbc06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8034000 tx=0x7f4bc802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 msgr2=0x7f4bd00c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c430 tx=0x7f4bc400c8f0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 shutdown_connections 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bbc06be30 0x7f4bbc06e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bd015eef0 0x7f4bd00cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bd0154f40 0x7f4bd00c7810 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 --2- 172.21.15.49:0/2635900644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bd00cb290 0x7f4bd01f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:12.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 >> 172.21.15.49:0/2635900644 conn(0x7f4bd0067b80 msgr2=0x7f4bd014a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 shutdown_connections 2024-05-09T08:56:12.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:12.931+0000 7f4bd77d5640 1 -- 172.21.15.49:0/2635900644 wait complete. 2024-05-09T08:56:15.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:56:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.046+0000 7f353b189640 1 Processor -- start 2024-05-09T08:56:16.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.046+0000 7f353b189640 1 -- start start 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f353b189640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3534154f40 0x7f353415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f353b189640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353415eef0 0x7f35340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f353b189640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f353415b1b0 con 0x7f35340cb290 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f353b189640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f35340b3fd0 con 0x7f3534154f40 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f353b189640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f353415bd30 con 0x7f353415eef0 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3534154f40 0x7f353415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3534154f40 0x7f353415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46866/0 (socket says 172.21.15.49:46866) 2024-05-09T08:56:16.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 -- 172.21.15.49:0/2304218421 learned_addr learned my addr 172.21.15.49:0/2304218421 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3539726640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353415eef0 0x7f35340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 -- 172.21.15.49:0/2304218421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353415eef0 msgr2=0x7f35340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353415eef0 0x7f35340cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 -- 172.21.15.49:0/2304218421 >> v1:172.21.15.183:6789/0 conn(0x7f35340cb290 legacy=0x7f35340cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.047+0000 7f3538f25640 1 -- 172.21.15.49:0/2304218421 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353415b490 con 0x7f3534154f40 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.048+0000 7f3538f25640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f353415e9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3528009870 tx=0x7f352802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c828c866ba59beee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.048+0000 7f35337fe640 1 -- 172.21.15.49:0/2304218421 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528037070 con 0x7f3534154f40 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.048+0000 7f35337fe640 1 -- 172.21.15.49:0/2304218421 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528032440 con 0x7f3534154f40 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.048+0000 7f35337fe640 1 -- 172.21.15.49:0/2304218421 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528032810 con 0x7f3534154f40 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 -- 172.21.15.49:0/2304218421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 msgr2=0x7f353415e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f353415e9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3528009870 tx=0x7f352802a1c0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 -- 172.21.15.49:0/2304218421 shutdown_connections 2024-05-09T08:56:16.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353415eef0 0x7f35340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 --2- 172.21.15.49:0/2304218421 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f353415e9b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 -- 172.21.15.49:0/2304218421 >> 172.21.15.49:0/2304218421 conn(0x7f3534067b80 msgr2=0x7f3534074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 -- 172.21.15.49:0/2304218421 shutdown_connections 2024-05-09T08:56:16.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.049+0000 7f353b189640 1 -- 172.21.15.49:0/2304218421 wait complete. 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.050+0000 7f353b189640 1 Processor -- start 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.050+0000 7f353b189640 1 -- start start 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 0x7f35340cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 0x7f3534143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f353415e490 con 0x7f3534154f40 2024-05-09T08:56:16.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f353415ba50 con 0x7f353415eef0 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f35340b3580 con 0x7f35340cb290 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46882/0 (socket says 172.21.15.49:46882) 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 -- 172.21.15.49:0/4023962997 learned_addr learned my addr 172.21.15.49:0/4023962997 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3539726640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 0x7f3534143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3538f25640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 0x7f35340cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 -- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 msgr2=0x7f35340cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 0x7f35340cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 -- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 msgr2=0x7f3534143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 0x7f3534143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 -- 172.21.15.49:0/4023962997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35340ca3a0 con 0x7f3534154f40 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3538f25640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 0x7f35340cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:16.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3539726640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 0x7f3534143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f3533fff640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f352000c3c0 tx=0x7f352000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35200052d0 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3520005490 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35340ca680 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.051+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520007260 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.052+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341f45c0 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.053+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353415ad50 con 0x7f3534154f40 2024-05-09T08:56:16.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.053+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3520007400 con 0x7f3534154f40 2024-05-09T08:56:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.054+0000 7f35317fa640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 0x7f351806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.054+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f352008a310 con 0x7f3534154f40 2024-05-09T08:56:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.054+0000 7f3538f25640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 0x7f351806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:16.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.054+0000 7f3538f25640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 0x7f351806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3528034000 tx=0x7f352802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:16.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.058+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f353415ad50 con 0x7f3534154f40 2024-05-09T08:56:16.132 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-05-09T08:56:16.133+0000 7f44b769a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T08:56:16.133 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-05-09T08:56:16.133+0000 7f44b769a640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-09T08:56:16.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.279+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f35340d1310 con 0x7f3534154f40 2024-05-09T08:56:16.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.279+0000 7f35317fa640 1 -- 172.21.15.49:0/4023962997 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f35340d1310 con 0x7f3534154f40 2024-05-09T08:56:16.279 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:16.279 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.281+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 msgr2=0x7f351806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.281+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 0x7f351806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3528034000 tx=0x7f352802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.281+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 msgr2=0x7f35341f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.281+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f352000c3c0 tx=0x7f352000c880 comp rx=0 tx=0).stop 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 shutdown_connections 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351806be30 0x7f351806e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353415eef0 0x7f3534143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3534154f40 0x7f35341f4ae0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 --2- 172.21.15.49:0/4023962997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35340cb290 0x7f35340cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:16.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 >> 172.21.15.49:0/4023962997 conn(0x7f3534067b80 msgr2=0x7f35340c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:16.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 shutdown_connections 2024-05-09T08:56:16.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:16.282+0000 7f353b189640 1 -- 172.21.15.49:0/4023962997 wait complete. 2024-05-09T08:56:17.722 INFO:tasks.ceph.mds.c.smithi049.stderr:2024-05-09T08:56:17.723+0000 7fe0b0331640 -1 quiesce.mds.1 failed to decode message of type 1285 v1: End of buffer [buffer:2] 2024-05-09T08:56:19.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:56:19.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.396+0000 7f8c1039e640 1 Processor -- start 2024-05-09T08:56:19.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.396+0000 7f8c1039e640 1 -- start start 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c1039e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c08154f40 0x7f8c0815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c1039e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8c0815eef0 0x7f8c080cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c1039e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c0815b1b0 con 0x7f8c080cb290 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c1039e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8c080b3fd0 con 0x7f8c08154f40 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c1039e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8c0815bd30 con 0x7f8c0815eef0 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c08154f40 0x7f8c0815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8c08154f40 0x7f8c0815e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46898/0 (socket says 172.21.15.49:46898) 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 -- 172.21.15.49:0/2007633748 learned_addr learned my addr 172.21.15.49:0/2007633748 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e93b640 1 --2- 172.21.15.49:0/2007633748 >> v2:172.21.15.183:3300/0 conn(0x7f8c0815eef0 0x7f8c080cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 -- 172.21.15.49:0/2007633748 >> v2:172.21.15.183:3300/0 conn(0x7f8c0815eef0 msgr2=0x7f8c080cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:19.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 --2- 172.21.15.49:0/2007633748 >> v2:172.21.15.183:3300/0 conn(0x7f8c0815eef0 0x7f8c080cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 -- 172.21.15.49:0/2007633748 >> v1:172.21.15.183:6789/0 conn(0x7f8c080cb290 legacy=0x7f8c080cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.397+0000 7f8c0e13a640 1 -- 172.21.15.49:0/2007633748 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0815b490 con 0x7f8c08154f40 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.398+0000 7f8c0e13a640 1 --2- 172.21.15.49:0/2007633748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c08154f40 0x7f8c0815e9b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8c080c56a0 tx=0x7f8c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c74849939c4adc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.398+0000 7f8c0d138640 1 -- 172.21.15.49:0/2007633748 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04037070 con 0x7f8c08154f40 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.398+0000 7f8c0d138640 1 -- 172.21.15.49:0/2007633748 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c04032440 con 0x7f8c08154f40 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.398+0000 7f8c0d138640 1 -- 172.21.15.49:0/2007633748 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c04032810 con 0x7f8c08154f40 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 -- 172.21.15.49:0/2007633748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c08154f40 msgr2=0x7f8c0815e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 --2- 172.21.15.49:0/2007633748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c08154f40 0x7f8c0815e9b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8c080c56a0 tx=0x7f8c0402a1e0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 -- 172.21.15.49:0/2007633748 shutdown_connections 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 --2- 172.21.15.49:0/2007633748 >> v2:172.21.15.183:3300/0 conn(0x7f8c0815eef0 0x7f8c080cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 --2- 172.21.15.49:0/2007633748 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c08154f40 0x7f8c0815e9b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 -- 172.21.15.49:0/2007633748 >> 172.21.15.49:0/2007633748 conn(0x7f8c08067b80 msgr2=0x7f8c08074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 -- 172.21.15.49:0/2007633748 shutdown_connections 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.399+0000 7f8c1039e640 1 -- 172.21.15.49:0/2007633748 wait complete. 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.400+0000 7f8c1039e640 1 Processor -- start 2024-05-09T08:56:19.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.400+0000 7f8c1039e640 1 -- start start 2024-05-09T08:56:19.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c080cb290 0x7f8c080c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 0x7f8c080cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0815e490 con 0x7f8c0815eef0 2024-05-09T08:56:19.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0815ba50 con 0x7f8c080cb290 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c1039e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8c080b3580 con 0x7f8c08154f40 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:46910/0 (socket says 172.21.15.49:46910) 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 -- 172.21.15.49:0/3212531395 learned_addr learned my addr 172.21.15.49:0/3212531395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e13a640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c080cb290 0x7f8c080c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0d939640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 0x7f8c080cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 -- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 msgr2=0x7f8c080cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 0x7f8c080cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 -- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c080cb290 msgr2=0x7f8c080c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c080cb290 0x7f8c080c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 -- 172.21.15.49:0/3212531395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c08149cc0 con 0x7f8c0815eef0 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0d939640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 0x7f8c080cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:19.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8c0e93b640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e70 tx=0x7f8c0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.401+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014620 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.402+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c00019070 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.402+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c080c7ab0 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.402+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014a20 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.402+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c08149020 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.403+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c08068560 con 0x7f8c0815eef0 2024-05-09T08:56:19.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.403+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c00014bc0 con 0x7f8c0815eef0 2024-05-09T08:56:19.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.404+0000 7f8bfeffd640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:19.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.404+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c0008b680 con 0x7f8c0815eef0 2024-05-09T08:56:19.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.404+0000 7f8c0e13a640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:19.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.405+0000 7f8c0e13a640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c04034000 tx=0x7f8c0402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:19.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.408+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c080c7ab0 con 0x7f8c0815eef0 2024-05-09T08:56:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.629+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8c080ca840 con 0x7f8c0815eef0 2024-05-09T08:56:19.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.630+0000 7f8bfeffd640 1 -- 172.21.15.49:0/3212531395 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f8c0005a0c0 con 0x7f8c0815eef0 2024-05-09T08:56:19.629 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:19.629 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 msgr2=0x7f8bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 0x7f8bec06e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8c04034000 tx=0x7f8c0402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 msgr2=0x7f8c080c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e70 tx=0x7f8c0000e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 shutdown_connections 2024-05-09T08:56:19.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8bec06be30 0x7f8bec06e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8c0815eef0 0x7f8c080c8ff0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8c08154f40 0x7f8c080cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 --2- 172.21.15.49:0/3212531395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c080cb290 0x7f8c080c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 >> 172.21.15.49:0/3212531395 conn(0x7f8c08067b80 msgr2=0x7f8c080c6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 shutdown_connections 2024-05-09T08:56:19.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:19.632+0000 7f8c1039e640 1 -- 172.21.15.49:0/3212531395 wait complete. 2024-05-09T08:56:21.133 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-05-09T08:56:21.133+0000 7f44b769a640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-09T08:56:22.642 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:56:22.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.746+0000 7f2ee1295640 1 Processor -- start 2024-05-09T08:56:22.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.746+0000 7f2ee1295640 1 -- start start 2024-05-09T08:56:22.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ee1295640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc154f40 0x7f2edc15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ee1295640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2edc15eef0 0x7f2edc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ee1295640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc15b1b0 con 0x7f2edc0cb290 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ee1295640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2edc0b3fd0 con 0x7f2edc15eef0 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ee1295640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc15bd30 con 0x7f2edc154f40 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ed3fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f2edc0cb290 0x7f2edc0cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48050/0 (socket says 172.21.15.49:48050) 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2ed3fff640 1 -- 172.21.15.49:0/2463129258 learned_addr learned my addr 172.21.15.49:0/2463129258 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2edaffd640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc154f40 0x7f2edc15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.747+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/2463129258 >> v2:172.21.15.183:3300/0 conn(0x7f2edc15eef0 0x7f2edc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996550144 0 0) 0x7f2edc15b1b0 con 0x7f2edc0cb290 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ec8003520 con 0x7f2edc0cb290 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2522720717 0 0) 0x7f2ec8003520 con 0x7f2edc0cb290 2024-05-09T08:56:22.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc154f40 msgr2=0x7f2edc15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc154f40 0x7f2edc15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 >> v2:172.21.15.183:3300/0 conn(0x7f2edc15eef0 msgr2=0x7f2edc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 --2- 172.21.15.49:0/2463129258 >> v2:172.21.15.183:3300/0 conn(0x7f2edc15eef0 0x7f2edc0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc15b490 con 0x7f2edc0cb290 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f2ec4001670 con 0x7f2edc0cb290 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 >> v1:172.21.15.49:6789/0 conn(0x7f2edc0cb290 legacy=0x7f2edc0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ec80039b0 0x7f2ec8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec80043a0 0x7f2ec8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec8004f90 0x7f2ec800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0b3fd0 con 0x7f2ec80043a0 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15bd30 con 0x7f2ec8004f90 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec800bf30 con 0x7f2ec80039b0 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2edaffd640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc154f40 0x7f2edc15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:22.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec80043a0 0x7f2ec8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec8004f90 0x7f2ec800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 -- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ec80039b0 msgr2=0x7f2ec8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ec80039b0 0x7f2ec8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 -- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec8004f90 msgr2=0x7f2ec800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec8004f90 0x7f2ec800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2ed3fff640 1 -- 172.21.15.49:0/2463129258 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec8003520 con 0x7f2ec80043a0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.748+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec8004f90 0x7f2ec800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.749+0000 7f2ed3fff640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec80043a0 0x7f2ec8004880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c240 tx=0x7f2ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e691d105441fb916 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.749+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4007d30 con 0x7f2ec80043a0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.749+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec4005ad0 con 0x7f2ec80043a0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.749+0000 7f2eda7fc640 1 -- 172.21.15.49:0/2463129258 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4007070 con 0x7f2ec80043a0 2024-05-09T08:56:22.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 -- 172.21.15.49:0/2463129258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec80043a0 msgr2=0x7f2ec8004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 --2- 172.21.15.49:0/2463129258 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2ec80043a0 0x7f2ec8004880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400c240 tx=0x7f2ec8005830 comp rx=0 tx=0).stop 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2edb7fe640 1 -- 172.21.15.49:0/2463129258 reap_dead start 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 -- 172.21.15.49:0/2463129258 shutdown_connections 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 -- 172.21.15.49:0/2463129258 >> 172.21.15.49:0/2463129258 conn(0x7f2edc067b80 msgr2=0x7f2edc074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 -- 172.21.15.49:0/2463129258 shutdown_connections 2024-05-09T08:56:22.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.750+0000 7f2ee1295640 1 -- 172.21.15.49:0/2463129258 wait complete. 2024-05-09T08:56:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.751+0000 7f2ee1295640 1 Processor -- start 2024-05-09T08:56:22.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.751+0000 7f2ee1295640 1 -- start start 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec80043a0 0x7f2edc1e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc14f9e0 0x7f2edc1f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15e490 con 0x7f2edc1f4b90 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15ba50 con 0x7f2ec80043a0 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc0b3580 con 0x7f2edc14f9e0 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47108/0 (socket says 172.21.15.49:47108) 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 -- 172.21.15.49:0/569133198 learned_addr learned my addr 172.21.15.49:0/569133198 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edaffd640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec80043a0 0x7f2edc1e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ed3fff640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc14f9e0 0x7f2edc1f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 -- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc14f9e0 msgr2=0x7f2edc1f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc14f9e0 0x7f2edc1f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 -- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec80043a0 msgr2=0x7f2edc1e7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec80043a0 0x7f2edc1e7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 -- 172.21.15.49:0/569133198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc0b4830 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2edb7fe640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009fd0 tx=0x7f2ed400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4016420 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed401b070 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc0b4b10 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40167c0 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.752+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc1e76d0 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.753+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc068560 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.754+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ed4005d10 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.754+0000 7f2ed37fe640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 0x7f2ebc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.755+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ed408b840 con 0x7f2edc1f4b90 2024-05-09T08:56:22.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.755+0000 7f2edaffd640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 0x7f2ebc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:22.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.755+0000 7f2edaffd640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 0x7f2ebc06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0047b0 tx=0x7f2ecc009590 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:22.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.759+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2ed4025020 con 0x7f2edc1f4b90 2024-05-09T08:56:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.982+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2edc0cdd20 con 0x7f2edc1f4b90 2024-05-09T08:56:22.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.983+0000 7f2ed37fe640 1 -- 172.21.15.49:0/569133198 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f2ed4059b10 con 0x7f2edc1f4b90 2024-05-09T08:56:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:22.982 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-05-09T08:56:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 msgr2=0x7f2ebc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 0x7f2ebc06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0047b0 tx=0x7f2ecc009590 comp rx=0 tx=0).stop 2024-05-09T08:56:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 msgr2=0x7f2edc1494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:22.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009fd0 tx=0x7f2ed400f520 comp rx=0 tx=0).stop 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 shutdown_connections 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ebc06be60 0x7f2ebc06e320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2edc1f4b90 0x7f2edc1494d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2edc14f9e0 0x7f2edc1f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 --2- 172.21.15.49:0/569133198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ec80043a0 0x7f2edc1e7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 >> 172.21.15.49:0/569133198 conn(0x7f2edc067b80 msgr2=0x7f2edc15a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 shutdown_connections 2024-05-09T08:56:22.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:22.985+0000 7f2ee1295640 1 -- 172.21.15.49:0/569133198 wait complete. 2024-05-09T08:56:25.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T08:56:26.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.099+0000 7fe897bf9640 1 Processor -- start 2024-05-09T08:56:26.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.099+0000 7fe897bf9640 1 -- start start 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe897bf9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 0x7fe8900cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe897bf9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8900ccf00 0x7fe8900cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe897bf9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe89015b1b0 con 0x7fe890154f40 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe897bf9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe8900b3fd0 con 0x7fe8900ccf00 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe897bf9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe89015bd30 con 0x7fe8900cb290 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe896196640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8900ccf00 0x7fe8900cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe895995640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe890154f40 0x7fe89015e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48066/0 (socket says 172.21.15.49:48066) 2024-05-09T08:56:26.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe895995640 1 -- 172.21.15.49:0/943053649 learned_addr learned my addr 172.21.15.49:0/943053649 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.100+0000 7fe895194640 1 --2- 172.21.15.49:0/943053649 >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 0x7fe8900cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe896196640 1 -- 172.21.15.49:0/943053649 >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 msgr2=0x7fe8900cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe896196640 1 --2- 172.21.15.49:0/943053649 >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 0x7fe8900cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe896196640 1 -- 172.21.15.49:0/943053649 >> v1:172.21.15.49:6789/0 conn(0x7fe890154f40 legacy=0x7fe89015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe896196640 1 -- 172.21.15.49:0/943053649 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89015b490 con 0x7fe8900ccf00 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe894993640 1 -- 172.21.15.49:0/943053649 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875154041 0 0) 0x7fe89015b1b0 con 0x7fe890154f40 2024-05-09T08:56:26.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe895194640 1 --2- 172.21.15.49:0/943053649 >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 0x7fe8900cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe896196640 1 --2- 172.21.15.49:0/943053649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900ccf00 0x7fe8900cd310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8900b3250 tx=0x7fe88802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6805618ed68c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.101+0000 7fe894993640 1 -- 172.21.15.49:0/943053649 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888007740 con 0x7fe8900ccf00 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe894993640 1 -- 172.21.15.49:0/943053649 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8880078e0 con 0x7fe8900ccf00 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe894993640 1 -- 172.21.15.49:0/943053649 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888007cb0 con 0x7fe8900ccf00 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 -- 172.21.15.49:0/943053649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900ccf00 msgr2=0x7fe8900cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 --2- 172.21.15.49:0/943053649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900ccf00 0x7fe8900cd310 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8900b3250 tx=0x7fe88802a5f0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 -- 172.21.15.49:0/943053649 shutdown_connections 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 --2- 172.21.15.49:0/943053649 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900ccf00 0x7fe8900cd310 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 --2- 172.21.15.49:0/943053649 >> v2:172.21.15.183:3300/0 conn(0x7fe8900cb290 0x7fe8900cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 -- 172.21.15.49:0/943053649 >> 172.21.15.49:0/943053649 conn(0x7fe890067b80 msgr2=0x7fe890074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 -- 172.21.15.49:0/943053649 shutdown_connections 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.102+0000 7fe897bf9640 1 -- 172.21.15.49:0/943053649 wait complete. 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.103+0000 7fe897bf9640 1 Processor -- start 2024-05-09T08:56:26.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.103+0000 7fe897bf9640 1 -- start start 2024-05-09T08:56:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8900ccf00 0x7fe8901f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe890154f40 0x7fe8901f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe89015e490 con 0x7fe8900cb290 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe89015ba50 con 0x7fe8900ccf00 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe897bf9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe8900b3580 con 0x7fe890154f40 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47140/0 (socket says 172.21.15.49:47140) 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 -- 172.21.15.49:0/1645433849 learned_addr learned my addr 172.21.15.49:0/1645433849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895194640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8900ccf00 0x7fe8901f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe896196640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe890154f40 0x7fe8901f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 -- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe890154f40 msgr2=0x7fe8901f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe890154f40 0x7fe8901f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 -- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8900ccf00 msgr2=0x7fe8901f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8900ccf00 0x7fe8901f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.104+0000 7fe895995640 1 -- 172.21.15.49:0/1645433849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8900d0220 con 0x7fe8900cb290 2024-05-09T08:56:26.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe895995640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c500 tx=0x7fe88c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c018070 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88c007580 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c004a20 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89015aab0 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.105+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8900cff40 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.106+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe890068560 con 0x7fe8900cb290 2024-05-09T08:56:26.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.106+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe88c01b070 con 0x7fe8900cb290 2024-05-09T08:56:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.107+0000 7fe8827fc640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 0x7fe87406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.107+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe88c08a160 con 0x7fe8900cb290 2024-05-09T08:56:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.107+0000 7fe895194640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 0x7fe87406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.108+0000 7fe895194640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 0x7fe87406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe890149500 tx=0x7fe884008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.111+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe88c058b10 con 0x7fe8900cb290 2024-05-09T08:56:26.133 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-05-09T08:56:26.134+0000 7f44b769a640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-09T08:56:26.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.335+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe8900cbd90 con 0x7fe8900cb290 2024-05-09T08:56:26.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.336+0000 7fe8827fc640 1 -- 172.21.15.49:0/1645433849 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe88c058410 con 0x7fe8900cb290 2024-05-09T08:56:26.335 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:26.335 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T08:56:26.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 msgr2=0x7fe87406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 0x7fe87406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe890149500 tx=0x7fe884008040 comp rx=0 tx=0).stop 2024-05-09T08:56:26.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 msgr2=0x7fe890148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c500 tx=0x7fe88c00c9c0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 shutdown_connections 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe87406be30 0x7fe87406e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe890154f40 0x7fe8901f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8900ccf00 0x7fe8901f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 --2- 172.21.15.49:0/1645433849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8900cb290 0x7fe890148c50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 >> 172.21.15.49:0/1645433849 conn(0x7fe890067b80 msgr2=0x7fe89014b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 shutdown_connections 2024-05-09T08:56:26.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.338+0000 7fe897bf9640 1 -- 172.21.15.49:0/1645433849 wait complete. 2024-05-09T08:56:26.348 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T08:56:26.348 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.452+0000 7f7c57d10640 1 Processor -- start 2024-05-09T08:56:26.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.452+0000 7f7c57d10640 1 -- start start 2024-05-09T08:56:26.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c57d10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c57d10640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c500cc190 0x7f7c500cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c57d10640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c500ccb60 con 0x7f7c50075200 2024-05-09T08:56:26.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c57d10640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7c50151bd0 con 0x7f7c500cc190 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c57d10640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5015e260 con 0x7f7c501517f0 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c500cc190 0x7f7c500cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c500cc190 0x7f7c500cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47142/0 (socket says 172.21.15.49:47142) 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 -- 172.21.15.49:0/514394721 learned_addr learned my addr 172.21.15.49:0/514394721 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c55aac640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 -- 172.21.15.49:0/514394721 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 msgr2=0x7f7c5015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 -- 172.21.15.49:0/514394721 >> v1:172.21.15.183:6789/0 conn(0x7f7c50075200 legacy=0x7f7c500ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c562ad640 1 -- 172.21.15.49:0/514394721 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5015e520 con 0x7f7c500cc190 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.453+0000 7f7c55aac640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.454+0000 7f7c562ad640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c500cc620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e80 tx=0x7f7c4802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdb995f36faef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.454+0000 7f7c54aaa640 1 -- 172.21.15.49:0/514394721 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48036070 con 0x7f7c500cc190 2024-05-09T08:56:26.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.454+0000 7f7c54aaa640 1 -- 172.21.15.49:0/514394721 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c48031600 con 0x7f7c500cc190 2024-05-09T08:56:26.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.454+0000 7f7c54aaa640 1 -- 172.21.15.49:0/514394721 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480319d0 con 0x7f7c500cc190 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 -- 172.21.15.49:0/514394721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 msgr2=0x7f7c500cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c500cc620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009e80 tx=0x7f7c4802a780 comp rx=0 tx=0).stop 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 -- 172.21.15.49:0/514394721 shutdown_connections 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c500cc620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 --2- 172.21.15.49:0/514394721 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 -- 172.21.15.49:0/514394721 >> 172.21.15.49:0/514394721 conn(0x7f7c5006bd00 msgr2=0x7f7c50074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 -- 172.21.15.49:0/514394721 shutdown_connections 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.455+0000 7f7c57d10640 1 -- 172.21.15.49:0/514394721 wait complete. 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.456+0000 7f7c57d10640 1 Processor -- start 2024-05-09T08:56:26.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.456+0000 7f7c57d10640 1 -- start start 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c50075200 0x7f7c501f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5015edc0 con 0x7f7c500cc190 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5015eae0 con 0x7f7c50075200 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c57d10640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5015f0a0 con 0x7f7c501517f0 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47148/0 (socket says 172.21.15.49:47148) 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 -- 172.21.15.49:0/1352513679 learned_addr learned my addr 172.21.15.49:0/1352513679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c55aac640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c50075200 0x7f7c501f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c562ad640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 -- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 msgr2=0x7f7c5014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5014c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 -- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c50075200 msgr2=0x7f7c501f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c50075200 0x7f7c501f0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 -- 172.21.15.49:0/1352513679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c500c7980 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c552ab640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c460 tx=0x7f7c4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40016020 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.457+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40012870 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.458+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5014efd0 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.458+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40012c60 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.458+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501f1850 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.459+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c50068560 con 0x7f7c500cc190 2024-05-09T08:56:26.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.460+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c40019020 con 0x7f7c500cc190 2024-05-09T08:56:26.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.460+0000 7f7c467fc640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.460+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c4008a840 con 0x7f7c500cc190 2024-05-09T08:56:26.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.461+0000 7f7c55aac640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.461+0000 7f7c55aac640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b840 tx=0x7f7c4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.465+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7c40020070 con 0x7f7c500cc190 2024-05-09T08:56:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.651+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c500caa00 con 0x7f7c500cc190 2024-05-09T08:56:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.652+0000 7f7c467fc640 1 -- 172.21.15.49:0/1352513679 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f7c500c7980 con 0x7f7c500cc190 2024-05-09T08:56:26.651 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:26.651 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.653+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 msgr2=0x7f7c3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.653+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b840 tx=0x7f7c4c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.653+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 msgr2=0x7f7c501f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.653+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c460 tx=0x7f7c4000c920 comp rx=0 tx=0).stop 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 shutdown_connections 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c501517f0 0x7f7c5014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500cc190 0x7f7c501f20b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 --2- 172.21.15.49:0/1352513679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c50075200 0x7f7c501f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 >> 172.21.15.49:0/1352513679 conn(0x7f7c5006bd00 msgr2=0x7f7c500c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 shutdown_connections 2024-05-09T08:56:26.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.654+0000 7f7c57d10640 1 -- 172.21.15.49:0/1352513679 wait complete. 2024-05-09T08:56:26.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:55:59.400583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:56:00.024628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:56:26.664 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:56:26.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.772+0000 7f0c90fa9640 1 Processor -- start 2024-05-09T08:56:26.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.772+0000 7f0c90fa9640 1 -- start start 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c90fa9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c8c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c90fa9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0c8c154f70 0x7f0c8c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c90fa9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8c072600 con 0x7f0c8c0c6640 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c90fa9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8c0728e0 con 0x7f0c8c154f70 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c90fa9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c0c5880 con 0x7f0c8c154b90 2024-05-09T08:56:26.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c8ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0c8c0c6640 0x7f0c8c0c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48082/0 (socket says 172.21.15.49:48082) 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c8ad9d640 1 -- 172.21.15.49:0/547339350 learned_addr learned my addr 172.21.15.49:0/547339350 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c8c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c83fff640 1 --2- 172.21.15.49:0/547339350 >> v2:172.21.15.183:3300/0 conn(0x7f0c8c154f70 0x7f0c8c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136422878 0 0) 0x7f0c8c072600 con 0x7f0c8c0c6640 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c78003520 con 0x7f0c8c0c6640 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2512689170 0 0) 0x7f0c78003520 con 0x7f0c8c0c6640 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 msgr2=0x7f0c8c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c8c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 >> v2:172.21.15.183:3300/0 conn(0x7f0c8c154f70 msgr2=0x7f0c8c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 --2- 172.21.15.49:0/547339350 >> v2:172.21.15.183:3300/0 conn(0x7f0c8c154f70 0x7f0c8c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c0c5b60 con 0x7f0c8c0c6640 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f0c840019f0 con 0x7f0c8c0c6640 2024-05-09T08:56:26.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 >> v1:172.21.15.49:6789/0 conn(0x7f0c8c0c6640 legacy=0x7f0c8c0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c780039b0 0x7f0c78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0728e0 con 0x7f0c78004d40 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0c5880 con 0x7f0c780039b0 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.773+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7800b9c0 con 0x7f0c8c154b90 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c780039b0 0x7f0c78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c83fff640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 -- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 msgr2=0x7f0c78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c154b90 0x7f0c78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 -- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c780039b0 msgr2=0x7f0c78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:26.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c780039b0 0x7f0c78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8ad9d640 1 -- 172.21.15.49:0/547339350 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c78003520 con 0x7f0c78004d40 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.774+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c780039b0 0x7f0c78003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c8ad9d640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c780051c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a880 tx=0x7f0c78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27038bb097825dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400a4a0 con 0x7f0c78004d40 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8402f070 con 0x7f0c78004d40 2024-05-09T08:56:26.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c89d9b640 1 -- 172.21.15.49:0/547339350 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8402bd00 con 0x7f0c78004d40 2024-05-09T08:56:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c90fa9640 1 -- 172.21.15.49:0/547339350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 msgr2=0x7f0c780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/547339350 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c780051c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a880 tx=0x7f0c78005830 comp rx=0 tx=0).stop 2024-05-09T08:56:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.775+0000 7f0c8ad9d640 1 -- 172.21.15.49:0/547339350 reap_dead start 2024-05-09T08:56:26.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.776+0000 7f0c90fa9640 1 -- 172.21.15.49:0/547339350 shutdown_connections 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.776+0000 7f0c90fa9640 1 -- 172.21.15.49:0/547339350 >> 172.21.15.49:0/547339350 conn(0x7f0c8c074840 msgr2=0x7f0c8c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.776+0000 7f0c90fa9640 1 -- 172.21.15.49:0/547339350 shutdown_connections 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.776+0000 7f0c90fa9640 1 -- 172.21.15.49:0/547339350 wait complete. 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.777+0000 7f0c90fa9640 1 Processor -- start 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.777+0000 7f0c90fa9640 1 -- start start 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 0x7f0c8c1488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c14c9f0 0x7f0c8c14ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c15af40 con 0x7f0c78004d40 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0c6120 con 0x7f0c8c0b45b0 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c0cb280 con 0x7f0c8c14c9f0 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47174/0 (socket says 172.21.15.49:47174) 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 -- 172.21.15.49:0/1074811915 learned_addr learned my addr 172.21.15.49:0/1074811915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8ad9d640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c14c9f0 0x7f0c8c14ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c83fff640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 0x7f0c8c1488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 -- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c14c9f0 msgr2=0x7f0c8c14ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c14c9f0 0x7f0c8c14ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 -- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 msgr2=0x7f0c8c1488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 0x7f0c8c1488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 -- 172.21.15.49:0/1074811915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c147a70 con 0x7f0c78004d40 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c83fff640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 0x7f0c8c1488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c8a59c640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c011a70 tx=0x7f0c7c011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00fbb0 con 0x7f0c78004d40 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00fd50 con 0x7f0c78004d40 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c14d3a0 con 0x7f0c78004d40 2024-05-09T08:56:26.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c014640 con 0x7f0c78004d40 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.778+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c14db20 con 0x7f0c78004d40 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.779+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c158f90 con 0x7f0c78004d40 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.780+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c7c012040 con 0x7f0c78004d40 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.780+0000 7f0c82ffd640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 0x7f0c6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.780+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c7c08b5f0 con 0x7f0c78004d40 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.781+0000 7f0c83fff640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 0x7f0c6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.781+0000 7f0c83fff640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 0x7f0c6c06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c15c040 tx=0x7f0c74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:26.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.785+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0c8c158f90 con 0x7f0c78004d40 2024-05-09T08:56:26.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.966+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c8c04f660 con 0x7f0c78004d40 2024-05-09T08:56:26.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.967+0000 7f0c82ffd640 1 -- 172.21.15.49:0/1074811915 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0c8c147a70 con 0x7f0c78004d40 2024-05-09T08:56:26.966 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-09T08:56:26.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 msgr2=0x7f0c6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 0x7f0c6c06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c15c040 tx=0x7f0c74008040 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 msgr2=0x7f0c8c1483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c011a70 tx=0x7f0c7c011f30 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 shutdown_connections 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0c6c06be30 0x7f0c6c06e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c8c14c9f0 0x7f0c8c14ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c8c0b45b0 0x7f0c8c1488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 --2- 172.21.15.49:0/1074811915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0c78004d40 0x7f0c8c1483a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 >> 172.21.15.49:0/1074811915 conn(0x7f0c8c074840 msgr2=0x7f0c8c158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 shutdown_connections 2024-05-09T08:56:26.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:26.969+0000 7f0c90fa9640 1 -- 172.21.15.49:0/1074811915 wait complete. 2024-05-09T08:56:26.979 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:56:27.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.086+0000 7f2aab02c640 1 Processor -- start 2024-05-09T08:56:27.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.086+0000 7f2aab02c640 1 -- start start 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aab02c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa4154b90 0x7f2aa40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aab02c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2aa4154f70 0x7f2aa40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aab02c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa4072600 con 0x7f2aa40c6640 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aab02c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa40728e0 con 0x7f2aa4154f70 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aab02c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa40c5880 con 0x7f2aa4154b90 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aa3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2aa4154f70 0x7f2aa40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aa3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2aa4154f70 0x7f2aa40cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47178/0 (socket says 172.21.15.49:47178) 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.087+0000 7f2aa3fff640 1 -- 172.21.15.49:0/1917314398 learned_addr learned my addr 172.21.15.49:0/1917314398 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.088+0000 7f2aa3fff640 1 -- 172.21.15.49:0/1917314398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa4154b90 msgr2=0x7f2aa40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:27.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.088+0000 7f2aa3fff640 1 --2- 172.21.15.49:0/1917314398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa4154b90 0x7f2aa40cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.088+0000 7f2aa3fff640 1 -- 172.21.15.49:0/1917314398 >> v1:172.21.15.183:6789/0 conn(0x7f2aa40c6640 legacy=0x7f2aa40c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:27.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.088+0000 7f2aa3fff640 1 -- 172.21.15.49:0/1917314398 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa40c5b60 con 0x7f2aa4154f70 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aa3fff640 1 --2- 172.21.15.49:0/1917314398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa40cb7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009870 tx=0x7f2a9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf44b68c136a1d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aa37fe640 1 -- 172.21.15.49:0/1917314398 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90037070 con 0x7f2aa4154f70 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aa37fe640 1 -- 172.21.15.49:0/1917314398 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a90032440 con 0x7f2aa4154f70 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aa37fe640 1 -- 172.21.15.49:0/1917314398 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90032810 con 0x7f2aa4154f70 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 -- 172.21.15.49:0/1917314398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 msgr2=0x7f2aa40cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1917314398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa40cb7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009870 tx=0x7f2a9002a2a0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 -- 172.21.15.49:0/1917314398 shutdown_connections 2024-05-09T08:56:27.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1917314398 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa40cb7c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1917314398 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa4154b90 0x7f2aa40cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 -- 172.21.15.49:0/1917314398 >> 172.21.15.49:0/1917314398 conn(0x7f2aa4074840 msgr2=0x7f2aa4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 -- 172.21.15.49:0/1917314398 shutdown_connections 2024-05-09T08:56:27.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.089+0000 7f2aab02c640 1 -- 172.21.15.49:0/1917314398 wait complete. 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.090+0000 7f2aab02c640 1 Processor -- start 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 -- start start 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa40c6640 0x7f2aa41e44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2aa4154b90 0x7f2aa4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa415af40 con 0x7f2aa4154f70 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa40c6120 con 0x7f2aa4154b90 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.091+0000 7f2aab02c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa4071820 con 0x7f2aa40c6640 2024-05-09T08:56:27.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47180/0 (socket says 172.21.15.49:47180) 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 -- 172.21.15.49:0/1967812805 learned_addr learned my addr 172.21.15.49:0/1967812805 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa8dc8640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa40c6640 0x7f2aa41e44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa3fff640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2aa4154b90 0x7f2aa4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 -- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa40c6640 msgr2=0x7f2aa41e44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa40c6640 0x7f2aa41e44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 -- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2aa4154b90 msgr2=0x7f2aa4150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2aa4154b90 0x7f2aa4150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 -- 172.21.15.49:0/1967812805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa41e49e0 con 0x7f2aa4154f70 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa95c9640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7b0 tx=0x7f2a9c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c017070 con 0x7f2aa4154f70 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a9c003b80 con 0x7f2aa4154f70 2024-05-09T08:56:27.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa414d260 con 0x7f2aa4154f70 2024-05-09T08:56:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c005070 con 0x7f2aa4154f70 2024-05-09T08:56:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.092+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa40b6010 con 0x7f2aa4154f70 2024-05-09T08:56:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.094+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa404f660 con 0x7f2aa4154f70 2024-05-09T08:56:27.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.094+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a9c01a070 con 0x7f2aa4154f70 2024-05-09T08:56:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.094+0000 7f2aa17fa640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.095+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2a9c08a360 con 0x7f2aa4154f70 2024-05-09T08:56:27.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.096+0000 7f2aa8dc8640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.098+0000 7f2aa8dc8640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 0x7f2a8806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009990 tx=0x7f2a98009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.099+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2a9c0586c0 con 0x7f2aa4154f70 2024-05-09T08:56:27.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.280+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2aa414dd00 con 0x7f2aa4154f70 2024-05-09T08:56:27.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.281+0000 7f2aa17fa640 1 -- 172.21.15.49:0/1967812805 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2a9c05c9c0 con 0x7f2aa4154f70 2024-05-09T08:56:27.280 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 msgr2=0x7f2a8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 0x7f2a8806e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009990 tx=0x7f2a98009110 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 msgr2=0x7f2aa4149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c7b0 tx=0x7f2a9c00cbb0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 shutdown_connections 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2a8806be30 0x7f2a8806e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2aa4154f70 0x7f2aa4149e70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2aa4154b90 0x7f2aa4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 --2- 172.21.15.49:0/1967812805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2aa40c6640 0x7f2aa41e44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 >> 172.21.15.49:0/1967812805 conn(0x7f2aa4074840 msgr2=0x7f2aa41538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 shutdown_connections 2024-05-09T08:56:27.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.283+0000 7f2aab02c640 1 -- 172.21.15.49:0/1967812805 wait complete. 2024-05-09T08:56:27.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:56:27.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 Processor -- start 2024-05-09T08:56:27.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 -- start start 2024-05-09T08:56:27.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb9c154b90 0x7fdb9c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 0x7fdb9c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c072600 con 0x7fdb9c0c6640 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.404+0000 7fdba1abd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c0728e0 con 0x7fdb9c154b90 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdba1abd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c0c5880 con 0x7fdb9c154f70 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb9c154b90 0x7fdb9c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdb9c154b90 0x7fdb9c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47186/0 (socket says 172.21.15.49:47186) 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 -- 172.21.15.49:0/3556364372 learned_addr learned my addr 172.21.15.49:0/3556364372 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9affd640 1 --2- 172.21.15.49:0/3556364372 >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 0x7fdb9c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 -- 172.21.15.49:0/3556364372 >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 msgr2=0x7fdb9c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 --2- 172.21.15.49:0/3556364372 >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 0x7fdb9c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 -- 172.21.15.49:0/3556364372 >> v1:172.21.15.183:6789/0 conn(0x7fdb9c0c6640 legacy=0x7fdb9c0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9b7fe640 1 -- 172.21.15.49:0/3556364372 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0c5b60 con 0x7fdb9c154b90 2024-05-09T08:56:27.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.405+0000 7fdb9affd640 1 --2- 172.21.15.49:0/3556364372 >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 0x7fdb9c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdb9b7fe640 1 --2- 172.21.15.49:0/3556364372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c0cb280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cdd20 tx=0x7fdb8c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8320149554222d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/3556364372 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c037070 con 0x7fdb9c154b90 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/3556364372 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c032440 con 0x7fdb9c154b90 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdb9a7fc640 1 -- 172.21.15.49:0/3556364372 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c032810 con 0x7fdb9c154b90 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 -- 172.21.15.49:0/3556364372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 msgr2=0x7fdb9c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 --2- 172.21.15.49:0/3556364372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c0cb280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0cdd20 tx=0x7fdb8c02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 -- 172.21.15.49:0/3556364372 shutdown_connections 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 --2- 172.21.15.49:0/3556364372 >> v2:172.21.15.183:3300/0 conn(0x7fdb9c154f70 0x7fdb9c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 --2- 172.21.15.49:0/3556364372 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c0cb280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 -- 172.21.15.49:0/3556364372 >> 172.21.15.49:0/3556364372 conn(0x7fdb9c074840 msgr2=0x7fdb9c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 -- 172.21.15.49:0/3556364372 shutdown_connections 2024-05-09T08:56:27.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.406+0000 7fdba1abd640 1 -- 172.21.15.49:0/3556364372 wait complete. 2024-05-09T08:56:27.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 Processor -- start 2024-05-09T08:56:27.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 -- start start 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb9c0c6640 0x7fdb9c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 0x7fdb9c0cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c15af40 con 0x7fdb9c154b90 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0c6120 con 0x7fdb9c154f70 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.408+0000 7fdba1abd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c071820 con 0x7fdb9c0c6640 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47196/0 (socket says 172.21.15.49:47196) 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 -- 172.21.15.49:0/1844172524 learned_addr learned my addr 172.21.15.49:0/1844172524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9b7fe640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb9c0c6640 0x7fdb9c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9bfff640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 0x7fdb9c0cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 -- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb9c0c6640 msgr2=0x7fdb9c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb9c0c6640 0x7fdb9c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 -- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 msgr2=0x7fdb9c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 0x7fdb9c0cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 -- 172.21.15.49:0/1844172524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1580d0 con 0x7fdb9c154b90 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9bfff640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 0x7fdb9c0cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb9affd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c7f0 tx=0x7fdb9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90015070 con 0x7fdb9c154b90 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90007820 con 0x7fdb9c154b90 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90007bd0 con 0x7fdb9c154b90 2024-05-09T08:56:27.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c1583b0 con 0x7fdb9c154b90 2024-05-09T08:56:27.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.409+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c156450 con 0x7fdb9c154b90 2024-05-09T08:56:27.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.410+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c04f660 con 0x7fdb9c154b90 2024-05-09T08:56:27.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.411+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb90018070 con 0x7fdb9c154b90 2024-05-09T08:56:27.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.411+0000 7fdb83fff640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.412+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdb9008afd0 con 0x7fdb9c154b90 2024-05-09T08:56:27.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.412+0000 7fdb9b7fe640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.412+0000 7fdb9b7fe640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c034000 tx=0x7fdb8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.416+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdb90059980 con 0x7fdb9c154b90 2024-05-09T08:56:27.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.597+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb9c158920 con 0x7fdb9c154b90 2024-05-09T08:56:27.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.598+0000 7fdb83fff640 1 -- 172.21.15.49:0/1844172524 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fdb90059280 con 0x7fdb9c154b90 2024-05-09T08:56:27.597 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-05-09T08:56:27.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 msgr2=0x7fdb8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 0x7fdb8806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c034000 tx=0x7fdb8c02e000 comp rx=0 tx=0).stop 2024-05-09T08:56:27.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 msgr2=0x7fdb9c157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c7f0 tx=0x7fdb9000ccb0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 shutdown_connections 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdb8806be30 0x7fdb8806e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9c154f70 0x7fdb9c0cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdb9c154b90 0x7fdb9c157540 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 --2- 172.21.15.49:0/1844172524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb9c0c6640 0x7fdb9c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 >> 172.21.15.49:0/1844172524 conn(0x7fdb9c074840 msgr2=0x7fdb9c0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 shutdown_connections 2024-05-09T08:56:27.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.600+0000 7fdba1abd640 1 -- 172.21.15.49:0/1844172524 wait complete. 2024-05-09T08:56:27.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.714+0000 7f467bee7640 1 Processor -- start 2024-05-09T08:56:27.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 -- start start 2024-05-09T08:56:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46740c7c10 0x7f467415c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4674152f60 0x7f467415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4674070b40 con 0x7f46740c7830 2024-05-09T08:56:27.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f467415c170 con 0x7f4674152f60 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467bee7640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f46740708a0 con 0x7f46740c7c10 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.715+0000 7f467a484640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4674152f60 0x7f467415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4674152f60 0x7f467415cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47212/0 (socket says 172.21.15.49:47212) 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 -- 172.21.15.49:0/3419816347 learned_addr learned my addr 172.21.15.49:0/3419816347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f4679482640 1 --2- 172.21.15.49:0/3419816347 >> v2:172.21.15.183:3300/0 conn(0x7f46740c7c10 0x7f467415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 -- 172.21.15.49:0/3419816347 >> v2:172.21.15.183:3300/0 conn(0x7f46740c7c10 msgr2=0x7f467415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 --2- 172.21.15.49:0/3419816347 >> v2:172.21.15.183:3300/0 conn(0x7f46740c7c10 0x7f467415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 -- 172.21.15.49:0/3419816347 >> v1:172.21.15.183:6789/0 conn(0x7f46740c7830 legacy=0x7f4674152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.716 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.716+0000 7f467a484640 1 -- 172.21.15.49:0/3419816347 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46740c5950 con 0x7f4674152f60 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467a484640 1 --2- 172.21.15.49:0/3419816347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4674152f60 0x7f467415cb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f46740c89c0 tx=0x7f466c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaea036b279f61d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f4678c81640 1 -- 172.21.15.49:0/3419816347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c036070 con 0x7f4674152f60 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f4678c81640 1 -- 172.21.15.49:0/3419816347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466c031600 con 0x7f4674152f60 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f4678c81640 1 -- 172.21.15.49:0/3419816347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0319d0 con 0x7f4674152f60 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 -- 172.21.15.49:0/3419816347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4674152f60 msgr2=0x7f467415cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 --2- 172.21.15.49:0/3419816347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4674152f60 0x7f467415cb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f46740c89c0 tx=0x7f466c02a730 comp rx=0 tx=0).stop 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 -- 172.21.15.49:0/3419816347 shutdown_connections 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 --2- 172.21.15.49:0/3419816347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4674152f60 0x7f467415cb40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 --2- 172.21.15.49:0/3419816347 >> v2:172.21.15.183:3300/0 conn(0x7f46740c7c10 0x7f467415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 -- 172.21.15.49:0/3419816347 >> 172.21.15.49:0/3419816347 conn(0x7f4674074840 msgr2=0x7f4674074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 -- 172.21.15.49:0/3419816347 shutdown_connections 2024-05-09T08:56:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.717+0000 7f467bee7640 1 -- 172.21.15.49:0/3419816347 wait complete. 2024-05-09T08:56:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 Processor -- start 2024-05-09T08:56:27.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 -- start start 2024-05-09T08:56:27.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46740c7830 0x7f4674154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 0x7f46740cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f46740c9990 con 0x7f46740c7c10 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46740c5ed0 con 0x7f4674152f60 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f467bee7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f46740c9b10 con 0x7f46740c7830 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f4679482640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f4679482640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47226/0 (socket says 172.21.15.49:47226) 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.719+0000 7f4679482640 1 -- 172.21.15.49:0/3078502201 learned_addr learned my addr 172.21.15.49:0/3078502201 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f467a484640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 0x7f46740cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 -- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46740c7830 msgr2=0x7f4674154e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:27.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46740c7830 0x7f4674154e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 -- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 msgr2=0x7f46740cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 0x7f46740cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 -- 172.21.15.49:0/3078502201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46741f49a0 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f467a484640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 0x7f46740cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f4679482640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f466400d800 tx=0x7f466400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400b6c0 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4664019070 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46741f3b40 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400ba80 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.720+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741f4100 con 0x7f46740c7c10 2024-05-09T08:56:27.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.721+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4674068770 con 0x7f46740c7c10 2024-05-09T08:56:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.722+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f466400bc20 con 0x7f46740c7c10 2024-05-09T08:56:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.722+0000 7f466a7fc640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 0x7f465806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.722+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f466408b2c0 con 0x7f46740c7c10 2024-05-09T08:56:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.722+0000 7f4679c83640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 0x7f465806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:27.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.723+0000 7f4679c83640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 0x7f465806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46700014c0 tx=0x7f4670007950 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:27.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.726+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4674068770 con 0x7f46740c7c10 2024-05-09T08:56:27.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.972+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f467406f3e0 con 0x7f46740c7c10 2024-05-09T08:56:27.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.973+0000 7f466a7fc640 1 -- 172.21.15.49:0/3078502201 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6908 (secure 0 0 0) 0x7f466401c360 con 0x7f46740c7c10 2024-05-09T08:56:27.972 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:27.972 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:24.751352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285,4241,4251,4238]},"id":1}]} 2024-05-09T08:56:27.973 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 msgr2=0x7f465806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 0x7f465806e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46700014c0 tx=0x7f4670007950 comp rx=0 tx=0).stop 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 msgr2=0x7f46740ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f466400d800 tx=0x7f466400dcc0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 shutdown_connections 2024-05-09T08:56:27.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f465806be30 0x7f465806e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4674152f60 0x7f46740cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f46740c7c10 0x7f46740ccae0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 --2- 172.21.15.49:0/3078502201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46740c7830 0x7f4674154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 >> 172.21.15.49:0/3078502201 conn(0x7f4674074840 msgr2=0x7f4674156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 shutdown_connections 2024-05-09T08:56:27.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:27.975+0000 7f467bee7640 1 -- 172.21.15.49:0/3078502201 wait complete. 2024-05-09T08:56:27.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.089+0000 7f0ee5088640 1 Processor -- start 2024-05-09T08:56:28.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.089+0000 7f0ee5088640 1 -- start start 2024-05-09T08:56:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.089+0000 7f0ee5088640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 0x7f0ee015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.089+0000 7f0ee5088640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0ee00cc190 0x7f0ee00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ee5088640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee00ccb60 con 0x7f0ee01517f0 2024-05-09T08:56:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ee5088640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee0151bd0 con 0x7f0ee00cc190 2024-05-09T08:56:28.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ee5088640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee015e260 con 0x7f0ee0075200 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0eded9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0ee01517f0 0x7f0ee00ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48084/0 (socket says 172.21.15.49:48084) 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0eded9d640 1 -- 172.21.15.49:0/2194754206 learned_addr learned my addr 172.21.15.49:0/2194754206 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ede59c640 1 --2- 172.21.15.49:0/2194754206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 0x7f0ee015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0edf59e640 1 --2- 172.21.15.49:0/2194754206 >> v2:172.21.15.183:3300/0 conn(0x7f0ee00cc190 0x7f0ee00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0eddd9b640 1 -- 172.21.15.49:0/2194754206 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103754428 0 0) 0x7f0ee00ccb60 con 0x7f0ee01517f0 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0eddd9b640 1 -- 172.21.15.49:0/2194754206 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ec8003520 con 0x7f0ee01517f0 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ede59c640 1 -- 172.21.15.49:0/2194754206 >> v2:172.21.15.183:3300/0 conn(0x7f0ee00cc190 msgr2=0x7f0ee00cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ede59c640 1 --2- 172.21.15.49:0/2194754206 >> v2:172.21.15.183:3300/0 conn(0x7f0ee00cc190 0x7f0ee00cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ede59c640 1 -- 172.21.15.49:0/2194754206 >> v1:172.21.15.49:6789/0 conn(0x7f0ee01517f0 legacy=0x7f0ee00ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.090+0000 7f0ede59c640 1 -- 172.21.15.49:0/2194754206 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee015e520 con 0x7f0ee0075200 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.091+0000 7f0ede59c640 1 --2- 172.21.15.49:0/2194754206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 0x7f0ee015b6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097d0 tx=0x7f0ed402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b753ad28cfe6ed1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.091+0000 7f0eddd9b640 1 -- 172.21.15.49:0/2194754206 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4037070 con 0x7f0ee0075200 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.091+0000 7f0eddd9b640 1 -- 172.21.15.49:0/2194754206 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed4032440 con 0x7f0ee0075200 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.091+0000 7f0eddd9b640 1 -- 172.21.15.49:0/2194754206 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed40327f0 con 0x7f0ee0075200 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 -- 172.21.15.49:0/2194754206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 msgr2=0x7f0ee015b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 --2- 172.21.15.49:0/2194754206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 0x7f0ee015b6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40097d0 tx=0x7f0ed402a250 comp rx=0 tx=0).stop 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 -- 172.21.15.49:0/2194754206 shutdown_connections 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 --2- 172.21.15.49:0/2194754206 >> v2:172.21.15.183:3300/0 conn(0x7f0ee00cc190 0x7f0ee00cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 --2- 172.21.15.49:0/2194754206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee0075200 0x7f0ee015b6c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 -- 172.21.15.49:0/2194754206 >> 172.21.15.49:0/2194754206 conn(0x7f0ee006bd00 msgr2=0x7f0ee0074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 -- 172.21.15.49:0/2194754206 shutdown_connections 2024-05-09T08:56:28.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.092+0000 7f0ee5088640 1 -- 172.21.15.49:0/2194754206 wait complete. 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.093+0000 7f0ee5088640 1 Processor -- start 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.093+0000 7f0ee5088640 1 -- start start 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee0075200 0x7f0ee01f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee01517f0 0x7f0ee015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee015edc0 con 0x7f0ee00cc190 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee015eae0 con 0x7f0ee0075200 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ee5088640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee015f0a0 con 0x7f0ee01517f0 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47238/0 (socket says 172.21.15.49:47238) 2024-05-09T08:56:28.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 -- 172.21.15.49:0/1399892595 learned_addr learned my addr 172.21.15.49:0/1399892595 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0eded9d640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee0075200 0x7f0ee01f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0edf59e640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee01517f0 0x7f0ee015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 -- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee01517f0 msgr2=0x7f0ee015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee01517f0 0x7f0ee015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 -- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee0075200 msgr2=0x7f0ee01f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee0075200 0x7f0ee01f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.094+0000 7f0ede59c640 1 -- 172.21.15.49:0/1399892595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee01f51f0 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ede59c640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4000c00 tx=0x7f0ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4041020 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed403d620 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee0159700 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4037040 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.095+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee015cb40 con 0x7f0ee00cc190 2024-05-09T08:56:28.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.096+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee00b6810 con 0x7f0ee00cc190 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.096+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ed4044020 con 0x7f0ee00cc190 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.097+0000 7f0ec77fe640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 0x7f0ecc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.097+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ed40ab420 con 0x7f0ee00cc190 2024-05-09T08:56:28.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.097+0000 7f0eded9d640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 0x7f0ecc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.097+0000 7f0eded9d640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 0x7f0ecc06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005880 tx=0x7f0ed0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.105+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0ee00b6810 con 0x7f0ee00cc190 2024-05-09T08:56:28.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.289+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ee00c6a90 con 0x7f0ee00cc190 2024-05-09T08:56:28.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.291+0000 7f0ec77fe640 1 -- 172.21.15.49:0/1399892595 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f0ee00c6a90 con 0x7f0ee00cc190 2024-05-09T08:56:28.290 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:28.290 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:28.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 msgr2=0x7f0ecc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 0x7f0ecc06e320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0005880 tx=0x7f0ed0009040 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 msgr2=0x7f0ee015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4000c00 tx=0x7f0ed40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 shutdown_connections 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0ecc06be60 0x7f0ecc06e320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ee01517f0 0x7f0ee015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ee00cc190 0x7f0ee015d060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 --2- 172.21.15.49:0/1399892595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ee0075200 0x7f0ee01f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 >> 172.21.15.49:0/1399892595 conn(0x7f0ee006bd00 msgr2=0x7f0ee0150aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 shutdown_connections 2024-05-09T08:56:28.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.293+0000 7f0ee5088640 1 -- 172.21.15.49:0/1399892595 wait complete. 2024-05-09T08:56:28.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:55:59.400583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:56:00.024628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:56:28.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:56:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.411+0000 7fc2b0929640 1 Processor -- start 2024-05-09T08:56:28.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.411+0000 7fc2b0929640 1 -- start start 2024-05-09T08:56:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2b0929640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2ac154f70 0x7fc2ac0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2b0929640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc2ac0c6b80 0x7fc2ac071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2b0929640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2ac0c6fd0 con 0x7fc2ac154b90 2024-05-09T08:56:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2b0929640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac072600 con 0x7fc2ac154f70 2024-05-09T08:56:28.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2b0929640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac0728e0 con 0x7fc2ac0c6b80 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2ac154f70 0x7fc2ac0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc2ac154f70 0x7fc2ac0c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47254/0 (socket says 172.21.15.49:47254) 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 -- 172.21.15.49:0/1148285700 learned_addr learned my addr 172.21.15.49:0/1148285700 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2aad9d640 1 --2- 172.21.15.49:0/1148285700 >> v2:172.21.15.183:3300/0 conn(0x7fc2ac0c6b80 0x7fc2ac071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 -- 172.21.15.49:0/1148285700 >> v2:172.21.15.183:3300/0 conn(0x7fc2ac0c6b80 msgr2=0x7fc2ac071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 --2- 172.21.15.49:0/1148285700 >> v2:172.21.15.183:3300/0 conn(0x7fc2ac0c6b80 0x7fc2ac071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 -- 172.21.15.49:0/1148285700 >> v1:172.21.15.183:6789/0 conn(0x7fc2ac154b90 legacy=0x7fc2ac0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.412+0000 7fc2a9d9b640 1 -- 172.21.15.49:0/1148285700 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0c5880 con 0x7fc2ac154f70 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.413+0000 7fc2a9d9b640 1 --2- 172.21.15.49:0/1148285700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac0c6640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0b4560 tx=0x7fc2a002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308ab90a84ca91a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.413+0000 7fc2a959a640 1 -- 172.21.15.49:0/1148285700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0037070 con 0x7fc2ac154f70 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.413+0000 7fc2a959a640 1 -- 172.21.15.49:0/1148285700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a0032440 con 0x7fc2ac154f70 2024-05-09T08:56:28.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.413+0000 7fc2a959a640 1 -- 172.21.15.49:0/1148285700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0032810 con 0x7fc2ac154f70 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 -- 172.21.15.49:0/1148285700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 msgr2=0x7fc2ac0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1148285700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac0c6640 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0b4560 tx=0x7fc2a002a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 -- 172.21.15.49:0/1148285700 shutdown_connections 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1148285700 >> v2:172.21.15.183:3300/0 conn(0x7fc2ac0c6b80 0x7fc2ac071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1148285700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac0c6640 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 -- 172.21.15.49:0/1148285700 >> 172.21.15.49:0/1148285700 conn(0x7fc2ac074840 msgr2=0x7fc2ac074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 -- 172.21.15.49:0/1148285700 shutdown_connections 2024-05-09T08:56:28.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.414+0000 7fc2b0929640 1 -- 172.21.15.49:0/1148285700 wait complete. 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.415+0000 7fc2b0929640 1 Processor -- start 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.415+0000 7fc2b0929640 1 -- start start 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 0x7fc2ac0cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 0x7fc2ac0b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0c6120 con 0x7fc2ac154f70 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac0c5e40 con 0x7fc2ac154b90 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2b0929640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac15af40 con 0x7fc2ac0c6b80 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47270/0 (socket says 172.21.15.49:47270) 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 -- 172.21.15.49:0/1689677370 learned_addr learned my addr 172.21.15.49:0/1689677370 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:28.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2a9d9b640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 0x7fc2ac0b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aa59c640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 0x7fc2ac0cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 -- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 msgr2=0x7fc2ac0cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 0x7fc2ac0cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 -- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 msgr2=0x7fc2ac0b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 0x7fc2ac0b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aad9d640 1 -- 172.21.15.49:0/1689677370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac14b1b0 con 0x7fc2ac154f70 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2a9d9b640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 0x7fc2ac0b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.416+0000 7fc2aa59c640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 0x7fc2ac0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc2aad9d640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007e90 tx=0x7fc2a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4014420 con 0x7fc2ac154f70 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a4019070 con 0x7fc2ac154f70 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac0cf8d0 con 0x7fc2ac154f70 2024-05-09T08:56:28.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4014800 con 0x7fc2ac154f70 2024-05-09T08:56:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.417+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac14d020 con 0x7fc2ac154f70 2024-05-09T08:56:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.418+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac069060 con 0x7fc2ac154f70 2024-05-09T08:56:28.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.418+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2a40149a0 con 0x7fc2ac154f70 2024-05-09T08:56:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.419+0000 7fc29affd640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 0x7fc28c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.419+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a408b670 con 0x7fc2ac154f70 2024-05-09T08:56:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.419+0000 7fc2aa59c640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 0x7fc28c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.419+0000 7fc2aa59c640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 0x7fc28c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a830 tx=0x7fc29c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.423+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc2ac069060 con 0x7fc2ac154f70 2024-05-09T08:56:28.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.609+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc2ac04f660 con 0x7fc2ac154f70 2024-05-09T08:56:28.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.610+0000 7fc29affd640 1 -- 172.21.15.49:0/1689677370 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fc2ac14b1b0 con 0x7fc2ac154f70 2024-05-09T08:56:28.609 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-09T08:56:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 msgr2=0x7fc28c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 0x7fc28c06e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00a830 tx=0x7fc29c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:28.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 msgr2=0x7fc2ac140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4007e90 tx=0x7fc2a400e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 shutdown_connections 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc28c06be30 0x7fc28c06e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc2ac154f70 0x7fc2ac140ec0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2ac154b90 0x7fc2ac0b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 --2- 172.21.15.49:0/1689677370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2ac0c6b80 0x7fc2ac0cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 >> 172.21.15.49:0/1689677370 conn(0x7fc2ac074840 msgr2=0x7fc2ac14a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 shutdown_connections 2024-05-09T08:56:28.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.612+0000 7fc2b0929640 1 -- 172.21.15.49:0/1689677370 wait complete. 2024-05-09T08:56:28.622 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:56:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.729+0000 7fb702445640 1 Processor -- start 2024-05-09T08:56:28.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.729+0000 7fb702445640 1 -- start start 2024-05-09T08:56:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb702445640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb6fc152f30 0x7fb6fc0c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb702445640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6fc0c9500 0x7fb6fc0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb702445640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6fc070990 con 0x7fb6fc0733c0 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb702445640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc0ca5f0 con 0x7fb6fc0c9500 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb702445640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb6fc15ccd0 con 0x7fb6fc152f30 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6fc0c9500 0x7fb6fc0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb6fc0c9500 0x7fb6fc0c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47282/0 (socket says 172.21.15.49:47282) 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 -- 172.21.15.49:0/725660896 learned_addr learned my addr 172.21.15.49:0/725660896 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fbfff640 1 --2- 172.21.15.49:0/725660896 >> v2:172.21.15.183:3300/0 conn(0x7fb6fc152f30 0x7fb6fc0c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 -- 172.21.15.49:0/725660896 >> v2:172.21.15.183:3300/0 conn(0x7fb6fc152f30 msgr2=0x7fb6fc0c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 --2- 172.21.15.49:0/725660896 >> v2:172.21.15.183:3300/0 conn(0x7fb6fc152f30 0x7fb6fc0c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 -- 172.21.15.49:0/725660896 >> v1:172.21.15.183:6789/0 conn(0x7fb6fc0733c0 legacy=0x7fb6fc072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:28.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.730+0000 7fb6fb7fe640 1 -- 172.21.15.49:0/725660896 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc15cfb0 con 0x7fb6fc0c9500 2024-05-09T08:56:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.731+0000 7fb6fb7fe640 1 --2- 172.21.15.49:0/725660896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0c7600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0cadf0 tx=0x7fb6f002e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a706f0a8d8691f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.731+0000 7fb6faffd640 1 -- 172.21.15.49:0/725660896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f003a070 con 0x7fb6fc0c9500 2024-05-09T08:56:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.731+0000 7fb6faffd640 1 -- 172.21.15.49:0/725660896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f0033710 con 0x7fb6fc0c9500 2024-05-09T08:56:28.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.731+0000 7fb6faffd640 1 -- 172.21.15.49:0/725660896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0033ae0 con 0x7fb6fc0c9500 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 -- 172.21.15.49:0/725660896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0c9500 msgr2=0x7fb6fc0c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 --2- 172.21.15.49:0/725660896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0c7600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0cadf0 tx=0x7fb6f002e6d0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 -- 172.21.15.49:0/725660896 shutdown_connections 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 --2- 172.21.15.49:0/725660896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0c7600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 --2- 172.21.15.49:0/725660896 >> v2:172.21.15.183:3300/0 conn(0x7fb6fc152f30 0x7fb6fc0c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 -- 172.21.15.49:0/725660896 >> 172.21.15.49:0/725660896 conn(0x7fb6fc0ce390 msgr2=0x7fb6fc0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 -- 172.21.15.49:0/725660896 shutdown_connections 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.732+0000 7fb702445640 1 -- 172.21.15.49:0/725660896 wait complete. 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.733+0000 7fb702445640 1 Processor -- start 2024-05-09T08:56:28.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.733+0000 7fb702445640 1 -- start start 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6fc152f30 0x7fb6fc1f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc0caa80 con 0x7fb6fc0733c0 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc0c5dd0 con 0x7fb6fc0c9500 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb702445640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb6fc156070 con 0x7fb6fc152f30 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47294/0 (socket says 172.21.15.49:47294) 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 -- 172.21.15.49:0/748161599 learned_addr learned my addr 172.21.15.49:0/748161599 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fb7fe640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb7009e2640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6fc152f30 0x7fb6fc1f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 -- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6fc152f30 msgr2=0x7fb6fc1f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6fc152f30 0x7fb6fc1f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 -- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6fc0c9500 msgr2=0x7fb6fc0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 -- 172.21.15.49:0/748161599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc15bf00 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6fbfff640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d620 tx=0x7fb6ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00b620 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.734+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ec019070 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.735+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00ba00 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.735+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc155460 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.735+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc155a20 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.736+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc151630 con 0x7fb6fc0733c0 2024-05-09T08:56:28.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.736+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6ec00bba0 con 0x7fb6fc0733c0 2024-05-09T08:56:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.736+0000 7fb6f8ff9640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 0x7fb6e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.737+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6ec08b1a0 con 0x7fb6fc0733c0 2024-05-09T08:56:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.737+0000 7fb6fb7fe640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 0x7fb6e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:28.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.737+0000 7fb6fb7fe640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 0x7fb6e806e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0037000 tx=0x7fb6f0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:28.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.745+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb6fc151630 con 0x7fb6fc0733c0 2024-05-09T08:56:28.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.940+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6fc04f660 con 0x7fb6fc0733c0 2024-05-09T08:56:28.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.940+0000 7fb6f8ff9640 1 -- 172.21.15.49:0/748161599 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fb6ec059be0 con 0x7fb6fc0733c0 2024-05-09T08:56:28.939 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-09T08:56:28.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 msgr2=0x7fb6e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 0x7fb6e806e320 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0037000 tx=0x7fb6f0008030 comp rx=0 tx=0).stop 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 msgr2=0x7fb6fc153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00d620 tx=0x7fb6ec00dae0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 shutdown_connections 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb6e806be60 0x7fb6e806e320 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6fc152f30 0x7fb6fc1f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6fc0c9500 0x7fb6fc0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 --2- 172.21.15.49:0/748161599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb6fc0733c0 0x7fb6fc153670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 >> 172.21.15.49:0/748161599 conn(0x7fb6fc0ce390 msgr2=0x7fb6fc150bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 shutdown_connections 2024-05-09T08:56:28.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:28.942+0000 7fb702445640 1 -- 172.21.15.49:0/748161599 wait complete. 2024-05-09T08:56:28.952 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:56:29.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.059+0000 7f7ae8fd5640 1 Processor -- start 2024-05-09T08:56:29.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.059+0000 7f7ae8fd5640 1 -- start start 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae8fd5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ae4154f70 0x7f7ae40c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae8fd5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7ae40c6b80 0x7f7ae4071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae8fd5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae40c6fd0 con 0x7f7ae4154b90 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae8fd5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae4072600 con 0x7f7ae4154f70 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae8fd5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae40728e0 con 0x7f7ae40c6b80 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ae4154f70 0x7f7ae40c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7ae4154f70 0x7f7ae40c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47310/0 (socket says 172.21.15.49:47310) 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 -- 172.21.15.49:0/3135109981 learned_addr learned my addr 172.21.15.49:0/3135109981 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7ae2d9d640 1 --2- 172.21.15.49:0/3135109981 >> v2:172.21.15.183:3300/0 conn(0x7f7ae40c6b80 0x7f7ae4071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 -- 172.21.15.49:0/3135109981 >> v2:172.21.15.183:3300/0 conn(0x7f7ae40c6b80 msgr2=0x7f7ae4071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 --2- 172.21.15.49:0/3135109981 >> v2:172.21.15.183:3300/0 conn(0x7f7ae40c6b80 0x7f7ae4071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 -- 172.21.15.49:0/3135109981 >> v1:172.21.15.183:6789/0 conn(0x7f7ae4154b90 legacy=0x7f7ae40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.060+0000 7f7adbfff640 1 -- 172.21.15.49:0/3135109981 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae40c5880 con 0x7f7ae4154f70 2024-05-09T08:56:29.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.061+0000 7f7adbfff640 1 --2- 172.21.15.49:0/3135109981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae40c6640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b4560 tx=0x7f7acc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18f719cba9df84d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.061+0000 7f7ae1d9b640 1 -- 172.21.15.49:0/3135109981 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc037070 con 0x7f7ae4154f70 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.061+0000 7f7ae1d9b640 1 -- 172.21.15.49:0/3135109981 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc032440 con 0x7f7ae4154f70 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.061+0000 7f7ae1d9b640 1 -- 172.21.15.49:0/3135109981 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc032810 con 0x7f7ae4154f70 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/3135109981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 msgr2=0x7f7ae40c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/3135109981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae40c6640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b4560 tx=0x7f7acc02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/3135109981 shutdown_connections 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/3135109981 >> v2:172.21.15.183:3300/0 conn(0x7f7ae40c6b80 0x7f7ae4071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/3135109981 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae40c6640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/3135109981 >> 172.21.15.49:0/3135109981 conn(0x7f7ae4074840 msgr2=0x7f7ae4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/3135109981 shutdown_connections 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.062+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/3135109981 wait complete. 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.063+0000 7f7ae8fd5640 1 Processor -- start 2024-05-09T08:56:29.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.063+0000 7f7ae8fd5640 1 -- start start 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 0x7f7ae40cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 0x7f7ae40b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae40c6120 con 0x7f7ae4154f70 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae40c5e40 con 0x7f7ae40c6b80 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae8fd5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae415af40 con 0x7f7ae4154b90 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47326/0 (socket says 172.21.15.49:47326) 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 -- 172.21.15.49:0/2192864801 learned_addr learned my addr 172.21.15.49:0/2192864801 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7adbfff640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 0x7f7ae40b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae259c640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 0x7f7ae40cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 -- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 msgr2=0x7f7ae40b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 0x7f7ae40b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 -- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 msgr2=0x7f7ae40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 0x7f7ae40cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae2d9d640 1 -- 172.21.15.49:0/2192864801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae414b1b0 con 0x7f7ae4154f70 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7adbfff640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 0x7f7ae40b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.064+0000 7f7ae259c640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 0x7f7ae40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7ae2d9d640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7adc007e90 tx=0x7f7adc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc014420 con 0x7f7ae4154f70 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc019070 con 0x7f7ae4154f70 2024-05-09T08:56:29.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc014800 con 0x7f7ae4154f70 2024-05-09T08:56:29.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae40cf8d0 con 0x7f7ae4154f70 2024-05-09T08:56:29.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.065+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae414d020 con 0x7f7ae4154f70 2024-05-09T08:56:29.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.066+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae4069060 con 0x7f7ae4154f70 2024-05-09T08:56:29.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.067+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7adc0149a0 con 0x7f7ae4154f70 2024-05-09T08:56:29.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.067+0000 7f7adaffd640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 0x7f7ac406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.067+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7adc08b670 con 0x7f7ae4154f70 2024-05-09T08:56:29.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.067+0000 7f7ae259c640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 0x7f7ac406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.068+0000 7f7ae259c640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 0x7f7ac406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400a830 tx=0x7f7ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.071+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ae4069060 con 0x7f7ae4154f70 2024-05-09T08:56:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.256+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ae40cbde0 con 0x7f7ae4154f70 2024-05-09T08:56:29.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.257+0000 7f7adaffd640 1 -- 172.21.15.49:0/2192864801 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7ae414b1b0 con 0x7f7ae4154f70 2024-05-09T08:56:29.256 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 msgr2=0x7f7ac406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 0x7f7ac406e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400a830 tx=0x7f7ad4008040 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 msgr2=0x7f7ae4140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7adc007e90 tx=0x7f7adc00e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 shutdown_connections 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7ac406be30 0x7f7ac406e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7ae4154f70 0x7f7ae4140ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ae4154b90 0x7f7ae40b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 --2- 172.21.15.49:0/2192864801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ae40c6b80 0x7f7ae40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 >> 172.21.15.49:0/2192864801 conn(0x7f7ae4074840 msgr2=0x7f7ae414a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 shutdown_connections 2024-05-09T08:56:29.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.259+0000 7f7ae8fd5640 1 -- 172.21.15.49:0/2192864801 wait complete. 2024-05-09T08:56:29.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:29.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.374+0000 7f159bb68640 1 Processor -- start 2024-05-09T08:56:29.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.374+0000 7f159bb68640 1 -- start start 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f159bb68640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f15941517f0 0x7f159415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f159bb68640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159415e260 con 0x7f1594075200 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f159bb68640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159415e520 con 0x7f15940cc190 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f159bb68640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1594151bd0 con 0x7f15941517f0 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f1599103640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1594075200 0x7f15940ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48090/0 (socket says 172.21.15.49:48090) 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f1599103640 1 -- 172.21.15.49:0/2115695481 learned_addr learned my addr 172.21.15.49:0/2115695481 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f1599904640 1 --2- 172.21.15.49:0/2115695481 >> v2:172.21.15.183:3300/0 conn(0x7f15941517f0 0x7f159415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3391004603 0 0) 0x7f159415e260 con 0x7f1594075200 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.375+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580003520 con 0x7f1594075200 2024-05-09T08:56:29.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 148366139 0 0) 0x7f1580003520 con 0x7f1594075200 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 >> v2:172.21.15.183:3300/0 conn(0x7f15941517f0 msgr2=0x7f159415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 --2- 172.21.15.49:0/2115695481 >> v2:172.21.15.183:3300/0 conn(0x7f15941517f0 0x7f159415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 >> v1:172.21.15.183:6789/0 conn(0x7f15940cc190 legacy=0x7f15940cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159415e800 con 0x7f1594075200 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f1584001670 con 0x7f1594075200 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 >> v1:172.21.15.49:6789/0 conn(0x7f1594075200 legacy=0x7f15940ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15800039b0 0x7f1580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1594151bd0 con 0x7f15800051c0 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f158000c070 con 0x7f1580004800 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f158000c270 con 0x7f15800039b0 2024-05-09T08:56:29.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1599103640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f1599904640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15800039b0 0x7f1580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 -- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15800039b0 msgr2=0x7f1580003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15800039b0 0x7f1580003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 -- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1580004800 msgr2=0x7f1580004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1580004800 0x7f1580004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.376+0000 7f159a105640 1 -- 172.21.15.49:0/2115695481 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1580003520 con 0x7f15800051c0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159a105640 1 -- 172.21.15.49:0/2115695481 reap_dead start 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159a105640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f158c001e50 tx=0x7f1580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae88883b513093eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c001a70 con 0x7f15800051c0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c003970 con 0x7f15800051c0 2024-05-09T08:56:29.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f1598902640 1 -- 172.21.15.49:0/2115695481 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c003d60 con 0x7f15800051c0 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159bb68640 1 -- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 msgr2=0x7f158000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159bb68640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f158c001e50 tx=0x7f1580005830 comp rx=0 tx=0).stop 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159bb68640 1 -- 172.21.15.49:0/2115695481 shutdown_connections 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159bb68640 1 --2- 172.21.15.49:0/2115695481 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f158000b9c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.377+0000 7f159bb68640 1 -- 172.21.15.49:0/2115695481 >> 172.21.15.49:0/2115695481 conn(0x7f159406bd00 msgr2=0x7f1594074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.378+0000 7f159bb68640 1 -- 172.21.15.49:0/2115695481 shutdown_connections 2024-05-09T08:56:29.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.378+0000 7f159bb68640 1 -- 172.21.15.49:0/2115695481 wait complete. 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.379+0000 7f159bb68640 1 Processor -- start 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.379+0000 7f159bb68640 1 -- start start 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15940c6250 0x7f1594143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15940b3b20 0x7f15941444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f159415f0a0 con 0x7f15800051c0 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f159415edc0 con 0x7f15940c6250 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f15940cbc70 con 0x7f15940b3b20 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47332/0 (socket says 172.21.15.49:47332) 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 -- 172.21.15.49:0/1566524034 learned_addr learned my addr 172.21.15.49:0/1566524034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599103640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15940c6250 0x7f1594143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159a105640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15940b3b20 0x7f15941444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 -- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15940b3b20 msgr2=0x7f15941444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15940b3b20 0x7f15941444a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 -- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15940c6250 msgr2=0x7f1594143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15940c6250 0x7f1594143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 -- 172.21.15.49:0/1566524034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159414bf80 con 0x7f15800051c0 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f1599904640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15900118e0 tx=0x7f1590011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159000fa90 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159000fc30 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159414c220 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590014440 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.380+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159415dcc0 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.381+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1594068560 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.382+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1590012040 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.382+0000 7f158a7fc640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 0x7f157806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.382+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f159008b610 con 0x7f15800051c0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.383+0000 7f1599103640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 0x7f157806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.383+0000 7f1599103640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 0x7f157806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15940c61e0 tx=0x7f1584009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.387+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1590019030 con 0x7f15800051c0 2024-05-09T08:56:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.612+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f159414c3f0 con 0x7f15800051c0 2024-05-09T08:56:29.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.612+0000 7f158a7fc640 1 -- 172.21.15.49:0/1566524034 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6908 (secure 0 0 0) 0x7f15900598e0 con 0x7f15800051c0 2024-05-09T08:56:29.611 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:29.612 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:24.751352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285,4241,4251,4238]},"id":1}]} 2024-05-09T08:56:29.612 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2024-05-09T08:56:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 msgr2=0x7f157806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 0x7f157806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15940c61e0 tx=0x7f1584009040 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 msgr2=0x7f159415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15900118e0 tx=0x7f1590011da0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 shutdown_connections 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f157806be30 0x7f157806e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15940b3b20 0x7f15941444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15940c6250 0x7f1594143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 --2- 172.21.15.49:0/1566524034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15800051c0 0x7f159415d780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 >> 172.21.15.49:0/1566524034 conn(0x7f159406bd00 msgr2=0x7f159415b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 shutdown_connections 2024-05-09T08:56:29.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.614+0000 7f159bb68640 1 -- 172.21.15.49:0/1566524034 wait complete. 2024-05-09T08:56:29.624 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:56:29.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.729+0000 7f50d0770640 1 Processor -- start 2024-05-09T08:56:29.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.729+0000 7f50d0770640 1 -- start start 2024-05-09T08:56:29.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50d0770640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f50c80cc8a0 0x7f50c815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50d0770640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c815e260 con 0x7f50c8075200 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50d0770640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c815e520 con 0x7f50c81517f0 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50d0770640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f50c8151bd0 con 0x7f50c80cc8a0 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50ced0d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f50c80cc8a0 0x7f50c815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50cdd0b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f50c8075200 0x7f50c80cc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48098/0 (socket says 172.21.15.49:48098) 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50cdd0b640 1 -- 172.21.15.49:0/3505159533 learned_addr learned my addr 172.21.15.49:0/3505159533 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50ced0d640 1 -- 172.21.15.49:0/3505159533 >> v1:172.21.15.183:6789/0 conn(0x7f50c81517f0 legacy=0x7f50c80ccfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50ced0d640 1 -- 172.21.15.49:0/3505159533 >> v1:172.21.15.49:6789/0 conn(0x7f50c8075200 legacy=0x7f50c80cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50cd50a640 1 -- 172.21.15.49:0/3505159533 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597162307 0 0) 0x7f50c815e260 con 0x7f50c8075200 2024-05-09T08:56:29.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.730+0000 7f50ced0d640 1 -- 172.21.15.49:0/3505159533 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c815e800 con 0x7f50c80cc8a0 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50ced0d640 1 --2- 172.21.15.49:0/3505159533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c80cc8a0 0x7f50c815b6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009d90 tx=0x7f50c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4280e00babe7f0a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50cd50a640 1 -- 172.21.15.49:0/3505159533 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007740 con 0x7f50c80cc8a0 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50cd50a640 1 -- 172.21.15.49:0/3505159533 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c00078e0 con 0x7f50c80cc8a0 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50cd50a640 1 -- 172.21.15.49:0/3505159533 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007cb0 con 0x7f50c80cc8a0 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50d0770640 1 -- 172.21.15.49:0/3505159533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c80cc8a0 msgr2=0x7f50c815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.731+0000 7f50d0770640 1 --2- 172.21.15.49:0/3505159533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c80cc8a0 0x7f50c815b6c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009d90 tx=0x7f50c002a730 comp rx=0 tx=0).stop 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.732+0000 7f50d0770640 1 -- 172.21.15.49:0/3505159533 shutdown_connections 2024-05-09T08:56:29.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.732+0000 7f50d0770640 1 --2- 172.21.15.49:0/3505159533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c80cc8a0 0x7f50c815b6c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.732+0000 7f50d0770640 1 -- 172.21.15.49:0/3505159533 >> 172.21.15.49:0/3505159533 conn(0x7f50c806bd00 msgr2=0x7f50c8074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.732+0000 7f50d0770640 1 -- 172.21.15.49:0/3505159533 shutdown_connections 2024-05-09T08:56:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.732+0000 7f50d0770640 1 -- 172.21.15.49:0/3505159533 wait complete. 2024-05-09T08:56:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 Processor -- start 2024-05-09T08:56:29.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 -- start start 2024-05-09T08:56:29.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50c8075200 0x7f50c80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c80cc8a0 0x7f50c80c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50c815f0a0 con 0x7f50c81517f0 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.733+0000 7f50d0770640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50c815edc0 con 0x7f50c80cc8a0 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50d0770640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f50c80cbc70 con 0x7f50c8075200 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47362/0 (socket says 172.21.15.49:47362) 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 -- 172.21.15.49:0/4173371134 learned_addr learned my addr 172.21.15.49:0/4173371134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ce50c640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50c8075200 0x7f50c80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50cdd0b640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c80cc8a0 0x7f50c80c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 -- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50c8075200 msgr2=0x7f50c80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50c8075200 0x7f50c80cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 -- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c80cc8a0 msgr2=0x7f50c80c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c80cc8a0 0x7f50c80c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 -- 172.21.15.49:0/4173371134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c81f4880 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50ced0d640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009eb0 tx=0x7f50c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0040070 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0031c40 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c81f4b60 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0003990 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.734+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c81f5120 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.735+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c814fc50 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.736+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50c0039040 con 0x7f50c81517f0 2024-05-09T08:56:29.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.736+0000 7f50beffd640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 0x7f50ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:29.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.736+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f50c00abe10 con 0x7f50c81517f0 2024-05-09T08:56:29.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.737+0000 7f50ce50c640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 0x7f50ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:29.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.737+0000 7f50ce50c640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 0x7f50ac06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c4009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:29.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.745+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50c814fc50 con 0x7f50c81517f0 2024-05-09T08:56:29.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.933+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50c8068560 con 0x7f50c81517f0 2024-05-09T08:56:29.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.934+0000 7f50beffd640 1 -- 172.21.15.49:0/4173371134 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16593 (secure 0 0 0) 0x7f50c007a170 con 0x7f50c81517f0 2024-05-09T08:56:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:29.934 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":20,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:56:02.682459+0000","last_up_change":"2024-05-09T08:55:55.016806+0000","last_in_change":"2024-05-09T08:55:46.620125+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6809","nonce":136930210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6811","nonce":136930210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6815","nonce":136930210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":136930210},{"type":"v1","addr":"172.21.15.49:6813","nonce":136930210}]},"public_addr":"172.21.15.49:6809/136930210","cluster_addr":"172.21.15.49:6811/136930210","heartbeat_back_addr":"172.21.15.49:6815/136930210","heartbeat_front_addr":"172.21.15.49:6813/136930210","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:56:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 msgr2=0x7f50ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 0x7f50ac06e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c4009060 comp rx=0 tx=0).stop 2024-05-09T08:56:29.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 msgr2=0x7f50c80ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009eb0 tx=0x7f50c0004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 shutdown_connections 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f50ac06be30 0x7f50ac06e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50c81517f0 0x7f50c80ce620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c80cc8a0 0x7f50c80c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 --2- 172.21.15.49:0/4173371134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50c8075200 0x7f50c80cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 >> 172.21.15.49:0/4173371134 conn(0x7f50c806bd00 msgr2=0x7f50c814ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 shutdown_connections 2024-05-09T08:56:29.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:29.936+0000 7f50d0770640 1 -- 172.21.15.49:0/4173371134 wait complete. 2024-05-09T08:56:29.946 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:30.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.050+0000 7f7bda810640 1 Processor -- start 2024-05-09T08:56:30.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.050+0000 7f7bda810640 1 -- start start 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bda810640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 0x7f7bd415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bda810640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7bd4075200 0x7f7bd40ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bda810640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd415e260 con 0x7f7bd40cd520 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bda810640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd4151bd0 con 0x7f7bd4075200 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bda810640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd415e520 con 0x7f7bd41517f0 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.051+0000 7f7bd37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7bd4075200 0x7f7bd40ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7bd4075200 0x7f7bd40ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47370/0 (socket says 172.21.15.49:47370) 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 -- 172.21.15.49:0/3288815242 learned_addr learned my addr 172.21.15.49:0/3288815242 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd3fff640 1 --2- 172.21.15.49:0/3288815242 >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 0x7f7bd415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:30.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 -- 172.21.15.49:0/3288815242 >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 msgr2=0x7f7bd415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 --2- 172.21.15.49:0/3288815242 >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 0x7f7bd415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 -- 172.21.15.49:0/3288815242 >> v1:172.21.15.183:6789/0 conn(0x7f7bd40cd520 legacy=0x7f7bd40cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 -- 172.21.15.49:0/3288815242 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd415e800 con 0x7f7bd4075200 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd3fff640 1 --2- 172.21.15.49:0/3288815242 >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 0x7f7bd415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.052+0000 7f7bd37fe640 1 --2- 172.21.15.49:0/3288815242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd4075200 0x7f7bd40ccfe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40c78a0 tx=0x7f7bc802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b640aec9cb072b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bd2ffd640 1 -- 172.21.15.49:0/3288815242 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8037070 con 0x7f7bd4075200 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bd2ffd640 1 -- 172.21.15.49:0/3288815242 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8032440 con 0x7f7bd4075200 2024-05-09T08:56:30.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bd2ffd640 1 -- 172.21.15.49:0/3288815242 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8032810 con 0x7f7bd4075200 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 -- 172.21.15.49:0/3288815242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd4075200 msgr2=0x7f7bd40ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 --2- 172.21.15.49:0/3288815242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd4075200 0x7f7bd40ccfe0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40c78a0 tx=0x7f7bc802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 -- 172.21.15.49:0/3288815242 shutdown_connections 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 --2- 172.21.15.49:0/3288815242 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd4075200 0x7f7bd40ccfe0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 --2- 172.21.15.49:0/3288815242 >> v2:172.21.15.183:3300/0 conn(0x7f7bd41517f0 0x7f7bd415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 -- 172.21.15.49:0/3288815242 >> 172.21.15.49:0/3288815242 conn(0x7f7bd406bd00 msgr2=0x7f7bd4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:30.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 -- 172.21.15.49:0/3288815242 shutdown_connections 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.053+0000 7f7bda810640 1 -- 172.21.15.49:0/3288815242 wait complete. 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.054+0000 7f7bda810640 1 Processor -- start 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 -- start start 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bd4075200 0x7f7bd41e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bd41517f0 0x7f7bd41598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd415f0a0 con 0x7f7bd40cd520 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd415edc0 con 0x7f7bd4075200 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bda810640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd40cbc70 con 0x7f7bd41517f0 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bd37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bd37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47386/0 (socket says 172.21.15.49:47386) 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.055+0000 7f7bd37fe640 1 -- 172.21.15.49:0/736053328 learned_addr learned my addr 172.21.15.49:0/736053328 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd8dad640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bd41517f0 0x7f7bd41598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:30.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 -- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bd41517f0 msgr2=0x7f7bd41598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bd41517f0 0x7f7bd41598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 -- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bd4075200 msgr2=0x7f7bd41e6320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bd4075200 0x7f7bd41e6320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 -- 172.21.15.49:0/736053328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd41e6a90 con 0x7f7bd40cd520 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd37fe640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8000c00 tx=0x7f7bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8041020 con 0x7f7bd40cd520 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc8030970 con 0x7f7bd40cd520 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd41550b0 con 0x7f7bd40cd520 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8037040 con 0x7f7bd40cd520 2024-05-09T08:56:30.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.056+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd41f1720 con 0x7f7bd40cd520 2024-05-09T08:56:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.057+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd4158620 con 0x7f7bd40cd520 2024-05-09T08:56:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.058+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bc8044020 con 0x7f7bd40cd520 2024-05-09T08:56:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.058+0000 7f7bd0ff9640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 0x7f7bc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.058+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7bc80ab8a0 con 0x7f7bd40cd520 2024-05-09T08:56:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.059+0000 7f7bd3fff640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 0x7f7bc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.059+0000 7f7bd3fff640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 0x7f7bc006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400a810 tx=0x7f7bc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:30.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.063+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7bc804b030 con 0x7f7bd40cd520 2024-05-09T08:56:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.284+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bd414fa70 con 0x7f7bd40cd520 2024-05-09T08:56:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.285+0000 7f7bd0ff9640 1 -- 172.21.15.49:0/736053328 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+6908 (secure 0 0 0) 0x7f7bc807a1c0 con 0x7f7bd40cd520 2024-05-09T08:56:30.284 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:30.285 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:24.751352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:creating","state_seq":1,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285,4241,4251,4238]},"id":1}]} 2024-05-09T08:56:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 15 2024-05-09T08:56:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 msgr2=0x7f7bc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 0x7f7bc006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400a810 tx=0x7f7bc4008040 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 msgr2=0x7f7bd414d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8000c00 tx=0x7f7bc80046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 shutdown_connections 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7bc006be30 0x7f7bc006e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7bd41517f0 0x7f7bd41598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7bd40cd520 0x7f7bd414d260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 --2- 172.21.15.49:0/736053328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bd4075200 0x7f7bd41e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 >> 172.21.15.49:0/736053328 conn(0x7f7bd406bd00 msgr2=0x7f7bd415b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 shutdown_connections 2024-05-09T08:56:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:30.287+0000 7f7bda810640 1 -- 172.21.15.49:0/736053328 wait complete. 2024-05-09T08:56:30.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T08:56:01.049338+0000', 'modified': '2024-05-09T08:56:24.751352+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4285, 'mds_1': 4251, 'mds_2': 4241, 'mds_3': 4238, 'mds_4': 4235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4235': {'gid': 4235, 'name': 'b', 'rank': 4, 'incarnation': 15, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.183:6838/2318987144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2318987144}, {'type': 'v1', 'addr': '172.21.15.183:6838', 'nonce': 2318987144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4238': {'gid': 4238, 'name': 'd', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6839/1999269737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6835', 'nonce': 1999269737}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1999269737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4241': {'gid': 4241, 'name': 'e', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.49:6837/2879019682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 2879019682}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 2879019682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6834/204294090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6833', 'nonce': 204294090}, {'type': 'v1', 'addr': '172.21.15.49:6834', 'nonce': 204294090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4285': {'gid': 4285, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6835/2800676488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6832', 'nonce': 2800676488}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2800676488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285, 4241, 4251, 4238]} 2024-05-09T08:56:30.902 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4235:up:creating 2024-05-09T08:56:31.133 INFO:tasks.ceph.mds.a.smithi049.stderr:2024-05-09T08:56:31.134+0000 7f44b769a640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-09T08:56:31.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:56:32.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.008+0000 7fb55ac38640 1 Processor -- start 2024-05-09T08:56:32.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.008+0000 7fb55ac38640 1 -- start start 2024-05-09T08:56:32.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb55ac38640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 0x7fb55415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb55ac38640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5540cc190 0x7fb5540cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb55ac38640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5540ccb20 con 0x7fb5541517f0 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb55ac38640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb554151bd0 con 0x7fb5540cc190 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb55ac38640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb55415e260 con 0x7fb554075200 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5540cc190 0x7fb5540cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb5540cc190 0x7fb5540cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47392/0 (socket says 172.21.15.49:47392) 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 -- 172.21.15.49:0/841041437 learned_addr learned my addr 172.21.15.49:0/841041437 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5589d4640 1 --1- 172.21.15.49:0/841041437 >> v1:172.21.15.49:6789/0 conn(0x7fb5541517f0 0x7fb5540ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:48100/0 (socket says 172.21.15.49:48100) 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb553fff640 1 --2- 172.21.15.49:0/841041437 >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 0x7fb55415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 -- 172.21.15.49:0/841041437 >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 msgr2=0x7fb55415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 --2- 172.21.15.49:0/841041437 >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 0x7fb55415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 -- 172.21.15.49:0/841041437 >> v1:172.21.15.49:6789/0 conn(0x7fb5541517f0 legacy=0x7fb5540ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5591d5640 1 -- 172.21.15.49:0/841041437 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55415e520 con 0x7fb5540cc190 2024-05-09T08:56:32.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb5537fe640 1 -- 172.21.15.49:0/841041437 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003999498 0 0) 0x7fb5540ccb20 con 0x7fb5541517f0 2024-05-09T08:56:32.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.009+0000 7fb553fff640 1 --2- 172.21.15.49:0/841041437 >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 0x7fb55415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:32.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.010+0000 7fb5591d5640 1 --2- 172.21.15.49:0/841041437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540cc5e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cdc20 tx=0x7fb54c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cc6185e2fe27ef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.010+0000 7fb5537fe640 1 -- 172.21.15.49:0/841041437 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c036070 con 0x7fb5540cc190 2024-05-09T08:56:32.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.010+0000 7fb5537fe640 1 -- 172.21.15.49:0/841041437 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54c031600 con 0x7fb5540cc190 2024-05-09T08:56:32.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.010+0000 7fb5537fe640 1 -- 172.21.15.49:0/841041437 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0319f0 con 0x7fb5540cc190 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 -- 172.21.15.49:0/841041437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 msgr2=0x7fb5540cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 --2- 172.21.15.49:0/841041437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540cc5e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5540cdc20 tx=0x7fb54c02a7c0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 -- 172.21.15.49:0/841041437 shutdown_connections 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 --2- 172.21.15.49:0/841041437 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540cc5e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 --2- 172.21.15.49:0/841041437 >> v2:172.21.15.183:3300/0 conn(0x7fb554075200 0x7fb55415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 -- 172.21.15.49:0/841041437 >> 172.21.15.49:0/841041437 conn(0x7fb55406bd00 msgr2=0x7fb554074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 -- 172.21.15.49:0/841041437 shutdown_connections 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.011+0000 7fb55ac38640 1 -- 172.21.15.49:0/841041437 wait complete. 2024-05-09T08:56:32.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.012+0000 7fb55ac38640 1 Processor -- start 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.012+0000 7fb55ac38640 1 -- start start 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb554075200 0x7fb5540d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 0x7fb5541f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb55415edc0 con 0x7fb5540cc190 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb55415eae0 con 0x7fb5541517f0 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb55ac38640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb55415f0a0 con 0x7fb554075200 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47402/0 (socket says 172.21.15.49:47402) 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 -- 172.21.15.49:0/489955742 learned_addr learned my addr 172.21.15.49:0/489955742 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb5591d5640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 0x7fb5541f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb5589d4640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb554075200 0x7fb5540d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 -- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb554075200 msgr2=0x7fb5540d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb554075200 0x7fb5540d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 -- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 msgr2=0x7fb5541f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 0x7fb5541f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb553fff640 1 -- 172.21.15.49:0/489955742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5540c91d0 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.013+0000 7fb5591d5640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 0x7fb5541f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb553fff640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c820 tx=0x7fb54000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540015070 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb540007a30 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5540cfe00 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540007e00 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.014+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5540c9470 con 0x7fb5540cc190 2024-05-09T08:56:32.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.015+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb554068560 con 0x7fb5540cc190 2024-05-09T08:56:32.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.015+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb540018070 con 0x7fb5540cc190 2024-05-09T08:56:32.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.016+0000 7fb5517fa640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 0x7fb53806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.016+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb54008aee0 con 0x7fb5540cc190 2024-05-09T08:56:32.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.016+0000 7fb5589d4640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 0x7fb53806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.017+0000 7fb5589d4640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 0x7fb53806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb548000f60 tx=0x7fb548007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.020+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb540059800 con 0x7fb5540cc190 2024-05-09T08:56:32.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.242+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5540caa00 con 0x7fb5540cc190 2024-05-09T08:56:32.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.243+0000 7fb5517fa640 1 -- 172.21.15.49:0/489955742 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6912 (secure 0 0 0) 0x7fb540059100 con 0x7fb5540cc190 2024-05-09T08:56:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:32.242 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:30.763288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4235,4238,4251,4241,4285]},"id":1}]} 2024-05-09T08:56:32.243 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 16 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 msgr2=0x7fb53806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 0x7fb53806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb548000f60 tx=0x7fb548007c00 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 msgr2=0x7fb5540ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c820 tx=0x7fb54000cc20 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 shutdown_connections 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb53806be30 0x7fb53806e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541517f0 0x7fb5541f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb5540cc190 0x7fb5540ca1b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 --2- 172.21.15.49:0/489955742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb554075200 0x7fb5540d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 >> 172.21.15.49:0/489955742 conn(0x7fb55406bd00 msgr2=0x7fb5540ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 shutdown_connections 2024-05-09T08:56:32.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.245+0000 7fb55ac38640 1 -- 172.21.15.49:0/489955742 wait complete. 2024-05-09T08:56:32.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T08:56:01.049338+0000', 'modified': '2024-05-09T08:56:30.763288+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4285, 'mds_1': 4251, 'mds_2': 4241, 'mds_3': 4238, 'mds_4': 4235}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4235': {'gid': 4235, 'name': 'b', 'rank': 4, 'incarnation': 15, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.183:6838/2318987144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2318987144}, {'type': 'v1', 'addr': '172.21.15.183:6838', 'nonce': 2318987144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4238': {'gid': 4238, 'name': 'd', 'rank': 3, 'incarnation': 13, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6839/1999269737', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6835', 'nonce': 1999269737}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1999269737}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4241': {'gid': 4241, 'name': 'e', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.49:6837/2879019682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6836', 'nonce': 2879019682}, {'type': 'v1', 'addr': '172.21.15.49:6837', 'nonce': 2879019682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.49:6834/204294090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6833', 'nonce': 204294090}, {'type': 'v1', 'addr': '172.21.15.49:6834', 'nonce': 204294090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4285': {'gid': 4285, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.49:6835/2800676488', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6832', 'nonce': 2800676488}, {'type': 'v1', 'addr': '172.21.15.49:6835', 'nonce': 2800676488}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4235, 4238, 4251, 4241, 4285]} 2024-05-09T08:56:32.860 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-09T08:56:32.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-09T08:56:32.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.966+0000 7f53fe491640 1 Processor -- start 2024-05-09T08:56:32.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.966+0000 7f53fe491640 1 -- start start 2024-05-09T08:56:32.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.966+0000 7f53fe491640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53f8154b90 0x7f53f80cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fe491640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 0x7f53f80cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fe491640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f8072600 con 0x7f53f80c6640 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fe491640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f53f80728e0 con 0x7f53f8154b90 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fe491640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f53f80c5880 con 0x7f53f8154f70 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53f8154b90 0x7f53f80cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fca2e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f53f80c6640 0x7f53f80c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35194/0 (socket says 172.21.15.49:35194) 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53fca2e640 1 -- 172.21.15.49:0/2808335820 learned_addr learned my addr 172.21.15.49:0/2808335820 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53effff640 1 --2- 172.21.15.49:0/2808335820 >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 0x7f53f80cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f7fff640 1 -- 172.21.15.49:0/2808335820 >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 msgr2=0x7f53f80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f7fff640 1 --2- 172.21.15.49:0/2808335820 >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 0x7f53f80cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f7fff640 1 -- 172.21.15.49:0/2808335820 >> v1:172.21.15.49:6789/0 conn(0x7f53f80c6640 legacy=0x7f53f80c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f7fff640 1 -- 172.21.15.49:0/2808335820 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f80c5b60 con 0x7f53f8154b90 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53f77fe640 1 -- 172.21.15.49:0/2808335820 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049511841 0 0) 0x7f53f8072600 con 0x7f53f80c6640 2024-05-09T08:56:32.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.967+0000 7f53effff640 1 --2- 172.21.15.49:0/2808335820 >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 0x7f53f80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53f7fff640 1 --2- 172.21.15.49:0/2808335820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154b90 0x7f53f80cb280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f53f80cdd20 tx=0x7f53e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4c1fe52c567834 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53f77fe640 1 -- 172.21.15.49:0/2808335820 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8037070 con 0x7f53f8154b90 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53f77fe640 1 -- 172.21.15.49:0/2808335820 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e8032440 con 0x7f53f8154b90 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53f77fe640 1 -- 172.21.15.49:0/2808335820 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8032810 con 0x7f53f8154b90 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 -- 172.21.15.49:0/2808335820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154b90 msgr2=0x7f53f80cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 --2- 172.21.15.49:0/2808335820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154b90 0x7f53f80cb280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f53f80cdd20 tx=0x7f53e802a230 comp rx=0 tx=0).stop 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 -- 172.21.15.49:0/2808335820 shutdown_connections 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 --2- 172.21.15.49:0/2808335820 >> v2:172.21.15.183:3300/0 conn(0x7f53f8154f70 0x7f53f80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 --2- 172.21.15.49:0/2808335820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154b90 0x7f53f80cb280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.968+0000 7f53fe491640 1 -- 172.21.15.49:0/2808335820 >> 172.21.15.49:0/2808335820 conn(0x7f53f8074840 msgr2=0x7f53f8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.969+0000 7f53fe491640 1 -- 172.21.15.49:0/2808335820 shutdown_connections 2024-05-09T08:56:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.969+0000 7f53fe491640 1 -- 172.21.15.49:0/2808335820 wait complete. 2024-05-09T08:56:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.970+0000 7f53fe491640 1 Processor -- start 2024-05-09T08:56:32.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.970+0000 7f53fe491640 1 -- start start 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.970+0000 7f53fe491640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53f80c6640 0x7f53f814cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.970+0000 7f53fe491640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f8154b90 0x7f53f81f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.970+0000 7f53fe491640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fe491640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53f815af40 con 0x7f53f8154f70 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fe491640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80c6120 con 0x7f53f8154b90 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fe491640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53f8071820 con 0x7f53f80c6640 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34136/0 (socket says 172.21.15.49:34136) 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 -- 172.21.15.49:0/3894924562 learned_addr learned my addr 172.21.15.49:0/3894924562 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53f80c6640 0x7f53f814cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53effff640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f8154b90 0x7f53f81f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 -- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53f80c6640 msgr2=0x7f53f814cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53f80c6640 0x7f53f814cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 -- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f8154b90 msgr2=0x7f53f81f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f8154b90 0x7f53f81f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 -- 172.21.15.49:0/3894924562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f814d4e0 con 0x7f53f8154f70 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fca2e640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c7a0 tx=0x7f53f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53f57fa640 1 -- 172.21.15.49:0/3894924562 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0018070 con 0x7f53f8154f70 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53f57fa640 1 -- 172.21.15.49:0/3894924562 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f000ce50 con 0x7f53f8154f70 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f80cf320 con 0x7f53f8154f70 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53f57fa640 1 -- 172.21.15.49:0/3894924562 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f000a820 con 0x7f53f8154f70 2024-05-09T08:56:32.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.971+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80cf8e0 con 0x7f53f8154f70 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.973+0000 7f53f57fa640 1 -- 172.21.15.49:0/3894924562 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53f000aa80 con 0x7f53f8154f70 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.974+0000 7f53f57fa640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 0x7f53d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.974+0000 7f53f57fa640 1 -- 172.21.15.49:0/3894924562 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53f008a4e0 con 0x7f53f8154f70 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.974+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 0x7f53d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.974+0000 7f5401195740 1 Processor -- start 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.974+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 0x7f53d806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53e8034000 tx=0x7f53e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 -- start start 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c341c0 0x55be74c4fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c06d50 0x55be74c08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55be74b6e4d0 con 0x55be74c091a0 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be74b163e0 con 0x55be74c341c0 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55be74b6a490 con 0x55be74c06d50 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f53fca2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f53fca2e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34138/0 (socket says 172.21.15.49:34138) 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f53fca2e640 1 -- 172.21.15.49:0/4161772674 learned_addr learned my addr 172.21.15.49:0/4161772674 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f53f7fff640 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c341c0 0x55be74c4fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.975+0000 7f53effff640 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c06d50 0x55be74c08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 -- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c06d50 msgr2=0x55be74c08c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c06d50 0x55be74c08c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 -- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c341c0 msgr2=0x55be74c4fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c341c0 0x55be74c4fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 -- 172.21.15.49:0/4161772674 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be748822d0 con 0x55be74c091a0 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53fca2e640 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53f0088a30 tx=0x7f53f005ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d439851bc7c13846 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53edffb640 1 -- 172.21.15.49:0/4161772674 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0056a30 con 0x55be74c091a0 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53edffb640 1 -- 172.21.15.49:0/4161772674 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f000ad70 con 0x55be74c091a0 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f53edffb640 1 -- 172.21.15.49:0/4161772674 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f008a0c0 con 0x55be74c091a0 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 -- 172.21.15.49:0/4161772674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 msgr2=0x55be74c345a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53f0088a30 tx=0x7f53f005ca30 comp rx=0 tx=0).stop 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 -- 172.21.15.49:0/4161772674 shutdown_connections 2024-05-09T08:56:32.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c091a0 0x55be74c345a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c06d50 0x55be74c08c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 --2- 172.21.15.49:0/4161772674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c341c0 0x55be74c4fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 -- 172.21.15.49:0/4161772674 >> 172.21.15.49:0/4161772674 conn(0x55be74c07be0 msgr2=0x55be74c4f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 -- 172.21.15.49:0/4161772674 shutdown_connections 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.976+0000 7f5401195740 1 -- 172.21.15.49:0/4161772674 wait complete. 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.977+0000 7f5401195740 1 -- start start 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.977+0000 7f5401195740 1 Processor -- start 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74c5fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74c31bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55be74c071e0 con 0x55be74c341c0 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be74b298e0 con 0x55be74c4f900 2024-05-09T08:56:32.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55be74c0d2c0 con 0x55be74c60090 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f5401195740 10 client.0 _ll_register_callbacks cb 0x55be74af69c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34150/0 (socket says 172.21.15.49:34150) 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 -- 172.21.15.49:0/4189778670 learned_addr learned my addr 172.21.15.49:0/4189778670 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53effff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74c5fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53fca2e640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74c31bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 -- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 msgr2=0x55be74c31bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74c31bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 -- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 msgr2=0x55be74c5fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74c5fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53f7fff640 1 -- 172.21.15.49:0/4189778670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be7494c6b0 con 0x55be74c341c0 2024-05-09T08:56:32.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53fca2e640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74c31bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.978+0000 7f53effff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74c5fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53f7fff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f53e8035040 tx=0x7f53e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504f259a3720057d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e804e020 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53e804a920 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f000ad70 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f53e8037040 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 10 client.4689 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55be74c325e0 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f000ad70 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f53e804ab20 con 0x55be74c341c0 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 10 client.4689 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:32.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 10 client.4689 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 20 client.4689 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74c5c670 0x55be74c6b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 4 client.4689 mds_command: new command op to 4235 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.979+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 --> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be74b7bc00 con 0x55be74c5c670 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.980+0000 7f53effff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74c5c670 0x55be74c6b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.980+0000 7f53d6ffd640 10 client.4689 ms_handle_connect on v2:172.21.15.183:6834/2318987144 2024-05-09T08:56:32.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:32.980+0000 7f53effff640 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74c5c670 0x55be74c6b3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25e8e360a1acc594 server_cookie=909b0bb5a3d1a2d6 in_seq=0 out_seq=0 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.770+0000 7f53d6ffd640 1 -- 172.21.15.49:0/4189778670 <== mds.4 v2:172.21.15.183:6834/2318987144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be74b7bc00 con 0x55be74c5c670 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.770+0000 7f53d6ffd640 10 client.4689 handle_command_reply: tid=0 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 client.4689 shutdown 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 msgr2=0x55be74c4d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f53e8035040 tx=0x7f53e8004730 comp rx=0 tx=0).stop 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 shutdown_connections 2024-05-09T08:56:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74c5c670 0x55be74c6b3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74c31bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74c5fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 --2- 172.21.15.49:0/4189778670 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4d9b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f53d6ffd640 0 client.4689 ms_handle_reset on v2:172.21.15.183:6834/2318987144 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 >> 172.21.15.49:0/4189778670 conn(0x55be74c091a0 msgr2=0x55be74c332e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 shutdown_connections 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 1 -- 172.21.15.49:0/4189778670 wait complete. 2024-05-09T08:56:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.771+0000 7f5401195740 20 client.4689 trim_cache size 0 max 16384 2024-05-09T08:56:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 Processor -- start 2024-05-09T08:56:35.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 -- start start 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c4f900 0x55be74c31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c60090 0x55be74c31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81f0840 con 0x55be74c341c0 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be74c4d1b0 con 0x55be74c60090 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55be74c4e100 con 0x55be74c4f900 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34152/0 (socket says 172.21.15.49:34152) 2024-05-09T08:56:35.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.790+0000 7f53f7fff640 1 -- 172.21.15.49:0/3970281743 learned_addr learned my addr 172.21.15.49:0/3970281743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 -- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c4f900 msgr2=0x55be74c31660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c4f900 0x55be74c31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 -- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c60090 msgr2=0x55be74c31ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c60090 0x55be74c31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 -- 172.21.15.49:0/3970281743 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be74c5fec0 con 0x55be74c341c0 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53e8032bb0 tx=0x7f53e8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e719d70d06878ff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53ed7fa640 1 -- 172.21.15.49:0/3970281743 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e804e040 con 0x55be74c341c0 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53ed7fa640 1 -- 172.21.15.49:0/3970281743 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e804a920 con 0x55be74c341c0 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.791+0000 7f53ed7fa640 1 -- 172.21.15.49:0/3970281743 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e803c440 con 0x55be74c341c0 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- 172.21.15.49:0/3970281743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 msgr2=0x55be74c4f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53e8032bb0 tx=0x7f53e8047000 comp rx=0 tx=0).stop 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- 172.21.15.49:0/3970281743 shutdown_connections 2024-05-09T08:56:35.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c60090 0x55be74c31ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c4f900 0x55be74c31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 --2- 172.21.15.49:0/3970281743 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c4f0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- 172.21.15.49:0/3970281743 >> 172.21.15.49:0/3970281743 conn(0x55be74c091a0 msgr2=0x55be74c459f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- 172.21.15.49:0/3970281743 shutdown_connections 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- 172.21.15.49:0/3970281743 wait complete. 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.792+0000 7f5401195740 1 -- start start 2024-05-09T08:56:35.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 Processor -- start 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74cc0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74cc13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x55be74c5c4f0 con 0x55be74c341c0 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be74c4fd30 con 0x55be74c4f900 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x55be74c5cb40 con 0x55be74c60090 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f5401195740 10 client.0 _ll_register_callbacks cb 0x7f53e8009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34160/0 (socket says 172.21.15.49:34160) 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.793+0000 7f53f7fff640 1 -- 172.21.15.49:0/3440594840 learned_addr learned my addr 172.21.15.49:0/3440594840 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53effff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74cc0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 -- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 msgr2=0x55be74cc13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74cc13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 -- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 msgr2=0x55be74cc0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74cc0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 -- 172.21.15.49:0/3440594840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be74c0a600 con 0x55be74c341c0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53f7fff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53e8032b50 tx=0x7f53e8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba26da0bb0d81f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e804e040 con 0x55be74c341c0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53e8067ac0 con 0x55be74c341c0 2024-05-09T08:56:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53e804a920 con 0x55be74c341c0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f53e8067cc0 con 0x55be74c341c0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 10 client.4695 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55be74c43f90 con 0x55be74c341c0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e804a920 con 0x55be74c341c0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f53e803c960 con 0x55be74c341c0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 10 client.4695 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 10 client.4695 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 20 client.4695 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74cca9d0 0x55be74ccadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 4 client.4695 mds_command: new command op to 4235 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.794+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 --> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] -- command(tid 0: {"prefix": "status"}) -- 0x55be74b75f50 con 0x55be74cca9d0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53effff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74cca9d0 0x55be74ccadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53effff640 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74cca9d0 0x55be74ccadb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9285d0505449ca9c server_cookie=30cea3cf598f3dd8 in_seq=0 out_seq=0 2024-05-09T08:56:35.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53d6ffd640 10 client.4695 ms_handle_connect on v2:172.21.15.183:6834/2318987144 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53d6ffd640 1 -- 172.21.15.49:0/3440594840 <== mds.4 v2:172.21.15.183:6834/2318987144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55be74b75f50 con 0x55be74cca9d0 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53d6ffd640 10 client.4695 handle_command_reply: tid=0 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 client.4695 shutdown 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 msgr2=0x55be74c0b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f53e8032b50 tx=0x7f53e8009190 comp rx=0 tx=0).stop 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 shutdown_connections 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:6834/2318987144,v1:172.21.15.183:6838/2318987144] conn(0x55be74cca9d0 0x55be74ccadb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55be74c60090 0x55be74cc13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f53d6ffd640 0 client.4695 ms_handle_reset on v2:172.21.15.183:6834/2318987144 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be74c4f900 0x55be74cc0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 --2- 172.21.15.49:0/3440594840 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x55be74c341c0 0x55be74c0b890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.795+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 >> 172.21.15.49:0/3440594840 conn(0x55be74c091a0 msgr2=0x55be74c56210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 shutdown_connections 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f5401195740 1 -- 172.21.15.49:0/3440594840 wait complete. 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f5401195740 20 client.4695 trim_cache size 0 max 16384 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 4, 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4235, 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-05-09T08:56:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 11.038931392, 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 16, 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 20, 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 40.673404222999999 2024-05-09T08:56:35.803 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 msgr2=0x7f53d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 0x7f53d806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f53e8034000 tx=0x7f53e802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 msgr2=0x7f53f8149d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c7a0 tx=0x7f53f000cc60 comp rx=0 tx=0).stop 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 shutdown_connections 2024-05-09T08:56:35.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f53d806be30 0x7f53d806e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53f8154f70 0x7f53f8149d70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53f8154b90 0x7f53f81f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 --2- 172.21.15.49:0/3894924562 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53f80c6640 0x7f53f814cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 >> 172.21.15.49:0/3894924562 conn(0x7f53f8074840 msgr2=0x7f53f80cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 shutdown_connections 2024-05-09T08:56:35.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.796+0000 7f53fe491640 1 -- 172.21.15.49:0/3894924562 wait complete. 2024-05-09T08:56:35.806 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T08:56:35.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.912+0000 7f71e57c1640 1 Processor -- start 2024-05-09T08:56:35.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.912+0000 7f71e57c1640 1 -- start start 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71e57c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e00c73a0 0x7f71e006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71e57c1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e00c4da0 con 0x7f71e0151be0 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71e57c1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e00c5060 con 0x7f71e015af60 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71e57c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f71e00c5c70 con 0x7f71e00c73a0 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71deffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f71e0151be0 0x7f71e00c45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35210/0 (socket says 172.21.15.49:35210) 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71deffd640 1 -- 172.21.15.49:0/1928177927 learned_addr learned my addr 172.21.15.49:0/1928177927 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732587198 0 0) 0x7f71e00c4da0 con 0x7f71e0151be0 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71df7fe640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e00c73a0 0x7f71e006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71cc003520 con 0x7f71e0151be0 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 596786360 0 0) 0x7f71cc003520 con 0x7f71e0151be0 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e00c73a0 msgr2=0x7f71e006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e00c73a0 0x7f71e006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.913+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 >> v1:172.21.15.183:6789/0 conn(0x7f71e015af60 legacy=0x7f71e00c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e00c5340 con 0x7f71e0151be0 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f71d0001670 con 0x7f71e0151be0 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 >> v1:172.21.15.49:6789/0 conn(0x7f71e0151be0 legacy=0x7f71e00c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc0039e0 0x7f71cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71cc004800 0x7f71cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00c5c70 con 0x7f71cc005180 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc00bf00 con 0x7f71cc0039e0 2024-05-09T08:56:35.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc00e2c0 con 0x7f71cc004800 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71deffd640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc0039e0 0x7f71cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71d7fff640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71cc004800 0x7f71cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 -- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71cc004800 msgr2=0x7f71cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71cc004800 0x7f71cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 -- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc0039e0 msgr2=0x7f71cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71cc0039e0 0x7f71cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.914+0000 7f71df7fe640 1 -- 172.21.15.49:0/1928177927 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc003520 con 0x7f71cc005180 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71df7fe640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71cc00b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009f90 tx=0x7f71cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad45aa8abca8347 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8009c10 con 0x7f71cc005180 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8036070 con 0x7f71cc005180 2024-05-09T08:56:35.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71de7fc640 1 -- 172.21.15.49:0/1928177927 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8031850 con 0x7f71cc005180 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71e57c1640 1 -- 172.21.15.49:0/1928177927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 msgr2=0x7f71cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71e57c1640 1 --2- 172.21.15.49:0/1928177927 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71cc00b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009f90 tx=0x7f71cc005830 comp rx=0 tx=0).stop 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71df7fe640 1 -- 172.21.15.49:0/1928177927 reap_dead start 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71e57c1640 1 -- 172.21.15.49:0/1928177927 shutdown_connections 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71e57c1640 1 -- 172.21.15.49:0/1928177927 >> 172.21.15.49:0/1928177927 conn(0x7f71e00cc290 msgr2=0x7f71e00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.915+0000 7f71e57c1640 1 -- 172.21.15.49:0/1928177927 shutdown_connections 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.916+0000 7f71e57c1640 1 -- 172.21.15.49:0/1928177927 wait complete. 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 Processor -- start 2024-05-09T08:56:35.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 -- start start 2024-05-09T08:56:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e00cbc20 0x7f71e0152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e0153650 0x7f71e0153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0151fc0 con 0x7f71cc005180 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71e00c5900 con 0x7f71e00cbc20 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71e57c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f71e0152140 con 0x7f71e0153650 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71deffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34186/0 (socket says 172.21.15.49:34186) 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.917+0000 7f71deffd640 1 -- 172.21.15.49:0/3105489901 learned_addr learned my addr 172.21.15.49:0/3105489901 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71df7fe640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e0153650 0x7f71e0153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71d7fff640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e00cbc20 0x7f71e0152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 -- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e0153650 msgr2=0x7f71e0153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e0153650 0x7f71e0153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 -- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e00cbc20 msgr2=0x7f71e0152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e00cbc20 0x7f71e0152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 -- 172.21.15.49:0/3105489901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e0154010 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71deffd640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71d00072f0 tx=0x7f71d00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71d77fe640 1 -- 172.21.15.49:0/3105489901 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000f9c0 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71d77fe640 1 -- 172.21.15.49:0/3105489901 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d0018070 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e00c3530 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71d77fe640 1 -- 172.21.15.49:0/3105489901 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d000fdc0 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.918+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e00caea0 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.920+0000 7f71d77fe640 1 -- 172.21.15.49:0/3105489901 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71d001d460 con 0x7f71cc005180 2024-05-09T08:56:35.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.920+0000 7f71d77fe640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 0x7f71c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.920+0000 7f71d77fe640 1 -- 172.21.15.49:0/3105489901 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71d008ae80 con 0x7f71cc005180 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.920+0000 7f71d7fff640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 0x7f71c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.921+0000 7f71d7fff640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 0x7f71c006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f71e0155530 tx=0x7f71c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.921+0000 7f71e85ac740 1 Processor -- start 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.921+0000 7f71e85ac740 1 -- start start 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 0x563bec289c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563bec1ef4d0 con 0x563bec28a1a0 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bec1973e0 con 0x563bec287d50 2024-05-09T08:56:35.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x563bec1eb490 con 0x563bec2b51c0 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34198/0 (socket says 172.21.15.49:34198) 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71df7fe640 1 -- 172.21.15.49:0/2245137379 learned_addr learned my addr 172.21.15.49:0/2245137379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71deffd640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71d7fff640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 0x563bec289c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.922+0000 7f71df7fe640 1 -- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 msgr2=0x563bec2d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71df7fe640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71df7fe640 1 -- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 msgr2=0x563bec289c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71df7fe640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 0x563bec289c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71df7fe640 1 -- 172.21.15.49:0/2245137379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bebf032d0 con 0x563bec28a1a0 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71d7fff640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 0x563bec289c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:35.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71df7fe640 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e230 tx=0x7f71d802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959e05d49cabd1f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71d4ff9640 1 -- 172.21.15.49:0/2245137379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8046630 con 0x563bec28a1a0 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71d4ff9640 1 -- 172.21.15.49:0/2245137379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8036040 con 0x563bec28a1a0 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71d4ff9640 1 -- 172.21.15.49:0/2245137379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8046a10 con 0x563bec28a1a0 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71e85ac740 1 -- 172.21.15.49:0/2245137379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 msgr2=0x563bec2b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.923+0000 7f71e85ac740 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f71d800e230 tx=0x7f71d802d000 comp rx=0 tx=0).stop 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 -- 172.21.15.49:0/2245137379 shutdown_connections 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28a1a0 0x563bec2b55a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec287d50 0x563bec289c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 --2- 172.21.15.49:0/2245137379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 -- 172.21.15.49:0/2245137379 >> 172.21.15.49:0/2245137379 conn(0x563bec288be0 msgr2=0x563bec2d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 -- 172.21.15.49:0/2245137379 shutdown_connections 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 -- 172.21.15.49:0/2245137379 wait complete. 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 -- start start 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.924+0000 7f71e85ac740 1 Processor -- start 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec2c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563bec2881e0 con 0x563bec28c3a0 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bec1aa8e0 con 0x563bec2c5aa0 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x563bec28e2c0 con 0x563bec2b51c0 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71e85ac740 10 client.0 _ll_register_callbacks cb 0x563bec1779c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:35.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34214/0 (socket says 172.21.15.49:34214) 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 -- 172.21.15.49:0/4023109024 learned_addr learned my addr 172.21.15.49:0/4023109024 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71deffd640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71df7fe640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec2c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 -- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 msgr2=0x563bec2c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 -- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 msgr2=0x563bec2c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec2c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.925+0000 7f71d7fff640 1 -- 172.21.15.49:0/4023109024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bebfcd6b0 con 0x563bec28c3a0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71c8015790 tx=0x7f71c8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f30b89ef2c09ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8019040 con 0x563bec28c3a0 2024-05-09T08:56:35.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71c8007240 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d8036040 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f71c8032050 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 10 client.4707 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563bec2e1160 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8036040 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f71c8035020 con 0x563bec28c3a0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 10 client.4707 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 10 client.4707 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 20 client.4707 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2c73b0 0x563bec2b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 4 client.4707 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 --> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bec1fcc00 con 0x563bec2c73b0 2024-05-09T08:56:35.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.926+0000 7f71deffd640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2c73b0 0x563bec2b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.927+0000 7f71bdffb640 10 client.4707 ms_handle_connect on v2:172.21.15.183:6835/1999269737 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.927+0000 7f71deffd640 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2c73b0 0x563bec2b3320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b45062b941c5849 server_cookie=f52bfcfd6130a766 in_seq=0 out_seq=0 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71bdffb640 1 -- 172.21.15.49:0/4023109024 <== mds.3 v2:172.21.15.183:6835/1999269737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bec1fcc00 con 0x563bec2c73b0 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71bdffb640 10 client.4707 handle_command_reply: tid=0 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 client.4707 shutdown 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 msgr2=0x563bec2dd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71c8015790 tx=0x7f71c8001120 comp rx=0 tx=0).stop 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 shutdown_connections 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec2c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec28c3a0 0x563bec2dd370 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2c73b0 0x563bec2b3320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4023109024 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2b51c0 0x563bec2c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71bdffb640 0 client.4707 ms_handle_reset on v2:172.21.15.183:6835/1999269737 2024-05-09T08:56:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 >> 172.21.15.49:0/4023109024 conn(0x563bec28a1a0 msgr2=0x563bec2cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 shutdown_connections 2024-05-09T08:56:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 1 -- 172.21.15.49:0/4023109024 wait complete. 2024-05-09T08:56:35.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.928+0000 7f71e85ac740 20 client.4707 trim_cache size 0 max 16384 2024-05-09T08:56:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.946+0000 7f71e85ac740 1 Processor -- start 2024-05-09T08:56:35.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.946+0000 7f71e85ac740 1 -- start start 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec28c3a0 0x563bec2cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec28b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01f90f0 con 0x563bec2b51c0 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ec770 con 0x563bec2c5aa0 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x563bec2ddac0 con 0x563bec28c3a0 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34228/0 (socket says 172.21.15.49:34228) 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 -- 172.21.15.49:0/462955620 learned_addr learned my addr 172.21.15.49:0/462955620 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 -- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec28c3a0 msgr2=0x563bec2cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec28c3a0 0x563bec2cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 -- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 msgr2=0x563bec28b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec28b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 -- 172.21.15.49:0/462955620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bec281290 con 0x563bec2b51c0 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.947+0000 7f71d7fff640 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000fb0 tx=0x7f71c8024070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c386edba90aa8076 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71bffff640 1 -- 172.21.15.49:0/462955620 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8019070 con 0x563bec2b51c0 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71bffff640 1 -- 172.21.15.49:0/462955620 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c801a040 con 0x563bec2b51c0 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71bffff640 1 -- 172.21.15.49:0/462955620 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c80074d0 con 0x563bec2b51c0 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 -- 172.21.15.49:0/462955620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 msgr2=0x563bec2cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f71c8000fb0 tx=0x7f71c8024070 comp rx=0 tx=0).stop 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 -- 172.21.15.49:0/462955620 shutdown_connections 2024-05-09T08:56:35.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec2c5aa0 0x563bec28b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec2cce80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 --2- 172.21.15.49:0/462955620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec28c3a0 0x563bec2cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 -- 172.21.15.49:0/462955620 >> 172.21.15.49:0/462955620 conn(0x563bec28a1a0 msgr2=0x563bec2c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 -- 172.21.15.49:0/462955620 shutdown_connections 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.948+0000 7f71e85ac740 1 -- 172.21.15.49:0/462955620 wait complete. 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 -- start start 2024-05-09T08:56:35.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 Processor -- start 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 0x563bec2da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 0x563bec345b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.949+0000 7f71e85ac740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x563bec2c77d0 con 0x563bec2b51c0 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bec2b31a0 con 0x563bec28c3a0 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x563bec2c7ea0 con 0x563bec2c5aa0 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 10 client.0 _ll_register_callbacks cb 0x7f71c8015440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34232/0 (socket says 172.21.15.49:34232) 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 -- 172.21.15.49:0/4082131760 learned_addr learned my addr 172.21.15.49:0/4082131760 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:35.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71df7fe640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 0x563bec345b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71deffd640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 0x563bec2da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 -- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 msgr2=0x563bec345b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 0x563bec345b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 -- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 msgr2=0x563bec2da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 0x563bec2da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 -- 172.21.15.49:0/4082131760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bec34fbf0 con 0x563bec2b51c0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71df7fe640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 0x563bec345b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71deffd640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 0x563bec2da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71d7fff640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71c8012690 tx=0x7f71c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d72c0909bd19cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8019040 con 0x563bec2b51c0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71c80067d0 con 0x563bec2b51c0 2024-05-09T08:56:35.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c801a040 con 0x563bec2b51c0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f71c804c050 con 0x563bec2b51c0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 10 client.4713 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563bec34b760 con 0x563bec2b51c0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.950+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c801a040 con 0x563bec2b51c0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7f71c8035040 con 0x563bec2b51c0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 10 client.4713 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 10 client.4713 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 20 client.4713 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2cbbb0 0x563bec358010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 4 client.4713 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 --> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] -- command(tid 0: {"prefix": "status"}) -- 0x563bec1f6f50 con 0x563bec2cbbb0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71deffd640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2cbbb0 0x563bec358010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:35.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71deffd640 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2cbbb0 0x563bec358010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df9fbbcce149a621 server_cookie=56ad49423a95943b in_seq=0 out_seq=0 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71bdffb640 10 client.4713 ms_handle_connect on v2:172.21.15.183:6835/1999269737 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71bdffb640 1 -- 172.21.15.49:0/4082131760 <== mds.3 v2:172.21.15.183:6835/1999269737 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563bec1f6f50 con 0x563bec2cbbb0 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71bdffb640 10 client.4713 handle_command_reply: tid=0 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.951+0000 7f71e85ac740 1 client.4713 shutdown 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 msgr2=0x563bec345640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71c8012690 tx=0x7f71c8017000 comp rx=0 tx=0).stop 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 shutdown_connections 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:6835/1999269737,v1:172.21.15.183:6839/1999269737] conn(0x563bec2cbbb0 0x563bec358010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x563bec2c5aa0 0x563bec345b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x563bec2b51c0 0x563bec345640 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71bdffb640 0 client.4713 ms_handle_reset on v2:172.21.15.183:6835/1999269737 2024-05-09T08:56:35.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 --2- 172.21.15.49:0/4082131760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bec28c3a0 0x563bec2da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 >> 172.21.15.49:0/4082131760 conn(0x563bec28a1a0 msgr2=0x563bec2f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 shutdown_connections 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 1 -- 172.21.15.49:0/4082131760 wait complete. 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e85ac740 20 client.4713 trim_cache size 0 max 16384 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 3, 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4238, 2024-05-09T08:56:35.959 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 17.211017651999999, 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 16, 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 20, 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 40.829477484999998 2024-05-09T08:56:35.960 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 msgr2=0x7f71c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 0x7f71c006e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f71e0155530 tx=0x7f71c8009040 comp rx=0 tx=0).stop 2024-05-09T08:56:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 msgr2=0x7f71e00cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f71d00072f0 tx=0x7f71d00077b0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 shutdown_connections 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f71c006be30 0x7f71c006e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f71e0153650 0x7f71e0153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e00cbc20 0x7f71e0152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 --2- 172.21.15.49:0/3105489901 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f71cc005180 0x7f71e00cb460 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 >> 172.21.15.49:0/3105489901 conn(0x7f71e00cc290 msgr2=0x7f71e0071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 shutdown_connections 2024-05-09T08:56:35.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:35.952+0000 7f71e57c1640 1 -- 172.21.15.49:0/3105489901 wait complete. 2024-05-09T08:56:35.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-09T08:56:36.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.068+0000 7fc45b058640 1 Processor -- start 2024-05-09T08:56:36.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.068+0000 7fc45b058640 1 -- start start 2024-05-09T08:56:36.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc45b058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc454158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc45b058640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc454072670 0x7fc4540c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc45b058640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc454159b10 con 0x7fc454152e30 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc45b058640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc454159270 con 0x7fc454072670 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc45b058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc454159550 con 0x7fc4540c7130 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc458df4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc454152e30 0x7fc4540c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35214/0 (socket says 172.21.15.49:35214) 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc458df4640 1 -- 172.21.15.49:0/3876077592 learned_addr learned my addr 172.21.15.49:0/3876077592 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc453fff640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc454158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3876077592 >> v2:172.21.15.183:3300/0 conn(0x7fc454072670 0x7fc4540c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3052610910 0 0) 0x7fc454159b10 con 0x7fc454152e30 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc444003520 con 0x7fc454152e30 2024-05-09T08:56:36.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1278971095 0 0) 0x7fc444003520 con 0x7fc454152e30 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 msgr2=0x7fc454158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc454158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 >> v2:172.21.15.183:3300/0 conn(0x7fc454072670 msgr2=0x7fc4540c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 --2- 172.21.15.49:0/3876077592 >> v2:172.21.15.183:3300/0 conn(0x7fc454072670 0x7fc4540c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454159830 con 0x7fc454152e30 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fc448001670 con 0x7fc454152e30 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.069+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 >> v1:172.21.15.49:6789/0 conn(0x7fc454152e30 legacy=0x7fc4540c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc4440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc444003f30 0x7fc444004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc444004d40 0x7fc4440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc454159270 con 0x7fc444004d40 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc454159550 con 0x7fc444003f30 2024-05-09T08:56:36.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc44400b9c0 con 0x7fc4540c7130 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc444004d40 0x7fc4440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc458df4640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc4440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc453fff640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc444003f30 0x7fc444004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 -- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 msgr2=0x7fc4440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540c7130 0x7fc4440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 -- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc444003f30 msgr2=0x7fc444004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc444003f30 0x7fc444004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.070+0000 7fc4595f5640 1 -- 172.21.15.49:0/3876077592 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc444003520 con 0x7fc444004d40 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc444004d40 0x7fc4440051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009fc0 tx=0x7fc444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c01e4297078102 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c009c10 con 0x7fc444004d40 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c036070 con 0x7fc444004d40 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc4537fe640 1 -- 172.21.15.49:0/3876077592 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c02d070 con 0x7fc444004d40 2024-05-09T08:56:36.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 -- 172.21.15.49:0/3876077592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc444004d40 msgr2=0x7fc4440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 --2- 172.21.15.49:0/3876077592 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc444004d40 0x7fc4440051c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009fc0 tx=0x7fc444005830 comp rx=0 tx=0).stop 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc4595f5640 1 -- 172.21.15.49:0/3876077592 reap_dead start 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 -- 172.21.15.49:0/3876077592 shutdown_connections 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 -- 172.21.15.49:0/3876077592 >> 172.21.15.49:0/3876077592 conn(0x7fc4540cc290 msgr2=0x7fc4540cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 -- 172.21.15.49:0/3876077592 shutdown_connections 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.071+0000 7fc45b058640 1 -- 172.21.15.49:0/3876077592 wait complete. 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 Processor -- start 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 -- start start 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540cad10 0x7fc4541f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454071390 0x7fc4541f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4540c6620 con 0x7fc4540cb0f0 2024-05-09T08:56:36.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4540c59f0 con 0x7fc454071390 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.073+0000 7fc45b058640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc45406b4b0 con 0x7fc4540cad10 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34250/0 (socket says 172.21.15.49:34250) 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 -- 172.21.15.49:0/3101932764 learned_addr learned my addr 172.21.15.49:0/3101932764 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 -- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540cad10 msgr2=0x7fc4541f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540cad10 0x7fc4541f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 -- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454071390 msgr2=0x7fc4541f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454071390 0x7fc4541f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 -- 172.21.15.49:0/3101932764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454070fb0 con 0x7fc4540cb0f0 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc453fff640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc440011910 tx=0x7fc440011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc4517fa640 1 -- 172.21.15.49:0/3101932764 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44000fa70 con 0x7fc4540cb0f0 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc4517fa640 1 -- 172.21.15.49:0/3101932764 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44000fc10 con 0x7fc4540cb0f0 2024-05-09T08:56:36.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc4517fa640 1 -- 172.21.15.49:0/3101932764 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440014440 con 0x7fc4540cb0f0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4541f1700 con 0x7fc4540cb0f0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.074+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4541f4720 con 0x7fc4540cb0f0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.076+0000 7fc4517fa640 1 -- 172.21.15.49:0/3101932764 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc440012040 con 0x7fc4540cb0f0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.076+0000 7fc4517fa640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 0x7fc43806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.076+0000 7fc4517fa640 1 -- 172.21.15.49:0/3101932764 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc44008b4e0 con 0x7fc4540cb0f0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.076+0000 7fc458df4640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 0x7fc43806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.077+0000 7fc458df4640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 0x7fc43806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc448004840 tx=0x7fc448001b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.077+0000 7fc45dd66740 1 Processor -- start 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.077+0000 7fc45dd66740 1 -- start start 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f011c0 0x557c98f1ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98ed3d50 0x557c98ed5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c98e3b4d0 con 0x557c98ed61a0 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c98de33e0 con 0x557c98f011c0 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x557c98e37490 con 0x557c98ed3d50 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34262/0 (socket says 172.21.15.49:34262) 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 -- 172.21.15.49:0/2667849132 learned_addr learned my addr 172.21.15.49:0/2667849132 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc453fff640 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98ed3d50 0x557c98ed5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc458df4640 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f011c0 0x557c98f1ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 -- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98ed3d50 msgr2=0x557c98ed5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98ed3d50 0x557c98ed5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 -- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f011c0 msgr2=0x557c98f1ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f011c0 0x557c98f1ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 -- 172.21.15.49:0/2667849132 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c98b4f2d0 con 0x557c98ed61a0 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.078+0000 7fc4595f5640 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e230 tx=0x7fc44c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079c01df083f6f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc43e7fc640 1 -- 172.21.15.49:0/2667849132 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003b90 con 0x557c98ed61a0 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc43e7fc640 1 -- 172.21.15.49:0/2667849132 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c036070 con 0x557c98ed61a0 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc43e7fc640 1 -- 172.21.15.49:0/2667849132 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c004020 con 0x557c98ed61a0 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 -- 172.21.15.49:0/2667849132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 msgr2=0x557c98f015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e230 tx=0x7fc44c004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 -- 172.21.15.49:0/2667849132 shutdown_connections 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98ed61a0 0x557c98f015a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98ed3d50 0x557c98ed5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 --2- 172.21.15.49:0/2667849132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f011c0 0x557c98f1ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 -- 172.21.15.49:0/2667849132 >> 172.21.15.49:0/2667849132 conn(0x557c98ed4be0 msgr2=0x557c98f1c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 -- 172.21.15.49:0/2667849132 shutdown_connections 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.079+0000 7fc45dd66740 1 -- 172.21.15.49:0/2667849132 wait complete. 2024-05-09T08:56:36.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.080+0000 7fc45dd66740 1 -- start start 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.080+0000 7fc45dd66740 1 Processor -- start 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f1f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98f1a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c98ed41e0 con 0x557c98f28990 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c98df68e0 con 0x557c98f2bf00 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x557c98eda2c0 con 0x557c98f011c0 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc45dd66740 10 client.0 _ll_register_callbacks cb 0x557c98dc39c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34264/0 (socket says 172.21.15.49:34264) 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 -- 172.21.15.49:0/942746845 learned_addr learned my addr 172.21.15.49:0/942746845 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc458df4640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f1f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc453fff640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98f1a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 -- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 msgr2=0x557c98f1f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f1f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 -- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 msgr2=0x557c98f1a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98f1a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 -- 172.21.15.49:0/942746845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c98c196b0 con 0x557c98f28990 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4595f5640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e200 tx=0x7fc44c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaf6ca71951eeae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4337fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.081+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003e20 con 0x557c98f28990 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc44c004020 con 0x557c98f28990 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44c036070 con 0x557c98f28990 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc44c056050 con 0x557c98f28990 2024-05-09T08:56:36.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 10 client.4725 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x557c98f1a2a0 con 0x557c98f28990 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c036070 con 0x557c98f28990 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fc44c059020 con 0x557c98f28990 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 10 client.4725 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 10 client.4725 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 20 client.4725 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f28e10 0x557c98f37900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 4 client.4725 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 --> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c98e48c00 con 0x557c98f28e10 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc458df4640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f28e10 0x557c98f37900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc458df4640 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f28e10 0x557c98f37900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bd73dad10c932c server_cookie=f52adf8b4eccce71 in_seq=0 out_seq=0 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.082+0000 7fc4337fe640 10 client.4725 ms_handle_connect on v2:172.21.15.49:6836/2879019682 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc4337fe640 1 -- 172.21.15.49:0/942746845 <== mds.2 v2:172.21.15.49:6836/2879019682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c98e48c00 con 0x557c98f28e10 2024-05-09T08:56:36.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc4337fe640 10 client.4725 handle_command_reply: tid=0 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 client.4725 shutdown 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 msgr2=0x557c98efe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e200 tx=0x7fc44c030000 comp rx=0 tx=0).stop 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 shutdown_connections 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f28e10 0x557c98f37900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98efe9b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98f1a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 --2- 172.21.15.49:0/942746845 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f1f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc4337fe640 0 client.4725 ms_handle_reset on v2:172.21.15.49:6836/2879019682 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 >> 172.21.15.49:0/942746845 conn(0x557c98ed61a0 msgr2=0x557c98ed7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.083+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 shutdown_connections 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.084+0000 7fc45dd66740 1 -- 172.21.15.49:0/942746845 wait complete. 2024-05-09T08:56:36.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.084+0000 7fc45dd66740 20 client.4725 trim_cache size 0 max 16384 2024-05-09T08:56:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 Processor -- start 2024-05-09T08:56:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 -- start start 2024-05-09T08:56:36.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98ed6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98effbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc4541e3e80 con 0x557c98f28990 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c98ed41e0 con 0x557c98f2bf00 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x557c98f1b180 con 0x557c98f011c0 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc453fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc453fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34268/0 (socket says 172.21.15.49:34268) 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.102+0000 7fc453fff640 1 -- 172.21.15.49:0/543136221 learned_addr learned my addr 172.21.15.49:0/543136221 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc4595f5640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98effbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 -- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 msgr2=0x557c98ed6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc458df4640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98ed6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98ed6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 -- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 msgr2=0x557c98effbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98effbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 -- 172.21.15.49:0/543136221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c98f1fc90 con 0x557c98f28990 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc4595f5640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98effbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc458df4640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98ed6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc453fff640 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc440055b60 tx=0x7fc440055ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb48980cc909ab7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc43dffb640 1 -- 172.21.15.49:0/543136221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440089070 con 0x557c98f28990 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc43dffb640 1 -- 172.21.15.49:0/543136221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440071030 con 0x557c98f28990 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc43dffb640 1 -- 172.21.15.49:0/543136221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440022040 con 0x557c98f28990 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 -- 172.21.15.49:0/543136221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 msgr2=0x557c98eff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc440055b60 tx=0x7fc440055ad0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 -- 172.21.15.49:0/543136221 shutdown_connections 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f2bf00 0x557c98effbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f28990 0x557c98eff670 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 --2- 172.21.15.49:0/543136221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98ed6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.103+0000 7fc45dd66740 1 -- 172.21.15.49:0/543136221 >> 172.21.15.49:0/543136221 conn(0x557c98ed61a0 msgr2=0x557c98f15310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.104+0000 7fc45dd66740 1 -- 172.21.15.49:0/543136221 shutdown_connections 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.104+0000 7fc45dd66740 1 -- 172.21.15.49:0/543136221 wait complete. 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.104+0000 7fc45dd66740 1 -- start start 2024-05-09T08:56:36.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.104+0000 7fc45dd66740 1 Processor -- start 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f28990 0x557c98f973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x557c98f1be70 con 0x557c98f2bf00 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c98f29ed0 con 0x557c98f28990 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x557c98f1c490 con 0x557c98f011c0 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc45dd66740 10 client.0 _ll_register_callbacks cb 0x7fc4480077d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34272/0 (socket says 172.21.15.49:34272) 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 -- 172.21.15.49:0/3620707393 learned_addr learned my addr 172.21.15.49:0/3620707393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc458df4640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc453fff640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f28990 0x557c98f973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 -- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 msgr2=0x557c98f2aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f2aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 -- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f28990 msgr2=0x557c98f973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f28990 0x557c98f973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.105+0000 7fc4595f5640 1 -- 172.21.15.49:0/3620707393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c98f97fe0 con 0x557c98f2bf00 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4595f5640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009c40 tx=0x7fc44c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320ea152f4c8d1a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c003e30 con 0x557c98f2bf00 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc44c003fd0 con 0x557c98f2bf00 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc440071030 con 0x557c98f2bf00 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc44c066050 con 0x557c98f2bf00 2024-05-09T08:56:36.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 10 client.4731 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x557c98f00220 con 0x557c98f2bf00 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440071030 con 0x557c98f2bf00 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fc44c069020 con 0x557c98f2bf00 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 10 client.4731 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 10 client.4731 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 20 client.4731 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f949e0 0x557c98fa0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 4 client.4731 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 --> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] -- command(tid 0: {"prefix": "status"}) -- 0x557c98e42f50 con 0x557c98f949e0 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.106+0000 7fc458df4640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f949e0 0x557c98fa0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc4337fe640 10 client.4731 ms_handle_connect on v2:172.21.15.49:6836/2879019682 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc458df4640 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f949e0 0x557c98fa0760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4d6b36423dfd42f server_cookie=4973e3f18925ca40 in_seq=0 out_seq=0 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc4337fe640 1 -- 172.21.15.49:0/3620707393 <== mds.2 v2:172.21.15.49:6836/2879019682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x557c98e42f50 con 0x557c98f949e0 2024-05-09T08:56:36.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc4337fe640 10 client.4731 handle_command_reply: tid=0 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 client.4731 shutdown 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 msgr2=0x557c98f97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009c40 tx=0x7fc44c004ab0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 shutdown_connections 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:6836/2879019682,v1:172.21.15.49:6837/2879019682] conn(0x557c98f949e0 0x557c98fa0760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x557c98f2bf00 0x557c98f97910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c98f28990 0x557c98f973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc4337fe640 0 client.4731 ms_handle_reset on v2:172.21.15.49:6836/2879019682 2024-05-09T08:56:36.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 --2- 172.21.15.49:0/3620707393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x557c98f011c0 0x557c98f2aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 >> 172.21.15.49:0/3620707393 conn(0x557c98ed61a0 msgr2=0x557c98f15100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 shutdown_connections 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 1 -- 172.21.15.49:0/3620707393 wait complete. 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.107+0000 7fc45dd66740 20 client.4731 trim_cache size 0 max 16384 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 2, 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4241, 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-05-09T08:56:36.115 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 23.381376600999999, 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 16, 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 20, 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 40.982277349999997 2024-05-09T08:56:36.116 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 msgr2=0x7fc43806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 0x7fc43806e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc448004840 tx=0x7fc448001b40 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 msgr2=0x7fc454070a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc440011910 tx=0x7fc440011dd0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 shutdown_connections 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc43806be30 0x7fc43806e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc454071390 0x7fc4541f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc4540cb0f0 0x7fc454070a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 --2- 172.21.15.49:0/3101932764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4540cad10 0x7fc4541f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 >> 172.21.15.49:0/3101932764 conn(0x7fc4540cc290 msgr2=0x7fc4540cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 shutdown_connections 2024-05-09T08:56:36.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.108+0000 7fc45b058640 1 -- 172.21.15.49:0/3101932764 wait complete. 2024-05-09T08:56:36.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T08:56:36.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.224+0000 7fc01962c640 1 Processor -- start 2024-05-09T08:56:36.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.224+0000 7fc01962c640 1 -- start start 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc01962c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc014155080 0x7fc014070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc01962c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc01415e4b0 0x7fc01415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc01962c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01415ee60 con 0x7fc0140c96f0 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc01962c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc0140707f0 con 0x7fc014155080 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc01962c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc0140c9fa0 con 0x7fc01415e4b0 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc012ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc014155080 0x7fc014070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc00a7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc0140c96f0 0x7fc01406b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35220/0 (socket says 172.21.15.49:35220) 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc00a7fc640 1 -- 172.21.15.49:0/1486026504 learned_addr learned my addr 172.21.15.49:0/1486026504 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc0137fe640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc01415e4b0 0x7fc01415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc012ffd640 1 -- 172.21.15.49:0/1486026504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc01415e4b0 msgr2=0x7fc01415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc012ffd640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc01415e4b0 0x7fc01415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc012ffd640 1 -- 172.21.15.49:0/1486026504 >> v1:172.21.15.49:6789/0 conn(0x7fc0140c96f0 legacy=0x7fc01406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc012ffd640 1 -- 172.21.15.49:0/1486026504 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0140ca280 con 0x7fc014155080 2024-05-09T08:56:36.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.225+0000 7fc0127fc640 1 -- 172.21.15.49:0/1486026504 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1843250646 0 0) 0x7fc01415ee60 con 0x7fc0140c96f0 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.226+0000 7fc012ffd640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc014155080 0x7fc014070140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0040097a0 tx=0x7fc00402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e1056bd6d43d94 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.226+0000 7fc0127fc640 1 -- 172.21.15.49:0/1486026504 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004037070 con 0x7fc014155080 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.226+0000 7fc0127fc640 1 -- 172.21.15.49:0/1486026504 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc004032440 con 0x7fc014155080 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.226+0000 7fc0127fc640 1 -- 172.21.15.49:0/1486026504 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004032810 con 0x7fc014155080 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 -- 172.21.15.49:0/1486026504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc014155080 msgr2=0x7fc014070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc014155080 0x7fc014070140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0040097a0 tx=0x7fc00402a180 comp rx=0 tx=0).stop 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 -- 172.21.15.49:0/1486026504 shutdown_connections 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc01415e4b0 0x7fc01415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 --2- 172.21.15.49:0/1486026504 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc014155080 0x7fc014070140 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 -- 172.21.15.49:0/1486026504 >> 172.21.15.49:0/1486026504 conn(0x7fc014072230 msgr2=0x7fc014072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 -- 172.21.15.49:0/1486026504 shutdown_connections 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.227+0000 7fc01962c640 1 -- 172.21.15.49:0/1486026504 wait complete. 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.228+0000 7fc01962c640 1 Processor -- start 2024-05-09T08:56:36.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.228+0000 7fc01962c640 1 -- start start 2024-05-09T08:56:36.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0140c96f0 0x7fc014155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 0x7fc0141590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0140cab20 con 0x7fc01415e4b0 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0140ca840 con 0x7fc0140c96f0 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc014155460 con 0x7fc014155080 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34284/0 (socket says 172.21.15.49:34284) 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 -- 172.21.15.49:0/102528936 learned_addr learned my addr 172.21.15.49:0/102528936 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc012ffd640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0140c96f0 0x7fc014155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 0x7fc0141590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 -- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 msgr2=0x7fc0141590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 0x7fc0141590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 -- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0140c96f0 msgr2=0x7fc014155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0140c96f0 0x7fc014155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 -- 172.21.15.49:0/102528936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0140c75a0 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 0x7fc0141590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc0137fe640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc00c007e40 tx=0x7fc00c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc00bfff640 1 -- 172.21.15.49:0/102528936 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c014620 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc00bfff640 1 -- 172.21.15.49:0/102528936 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c019070 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0141f0be0 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.229+0000 7fc00bfff640 1 -- 172.21.15.49:0/102528936 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c014a20 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.230+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141f11a0 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.231+0000 7fc00bfff640 1 -- 172.21.15.49:0/102528936 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc00c014bc0 con 0x7fc01415e4b0 2024-05-09T08:56:36.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.232+0000 7fc00bfff640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 0x7fbff406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.232+0000 7fc00bfff640 1 -- 172.21.15.49:0/102528936 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc00c08b770 con 0x7fc01415e4b0 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.232+0000 7fc012ffd640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 0x7fbff406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.232+0000 7fc012ffd640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 0x7fbff406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc004034000 tx=0x7fc00402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.232+0000 7fc01c40e740 1 Processor -- start 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 -- start start 2024-05-09T08:56:36.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7b97d50 0x556ac7b99c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 0x556ac7bc55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ac7aff4d0 con 0x556ac7bc51c0 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ac7aa73e0 con 0x556ac7b9a1a0 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x556ac7afb490 con 0x556ac7b97d50 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc012ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34290/0 (socket says 172.21.15.49:34290) 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.233+0000 7fc012ffd640 1 -- 172.21.15.49:0/3185215250 learned_addr learned my addr 172.21.15.49:0/3185215250 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc0137fe640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 0x556ac7bc55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7b97d50 0x556ac7b99c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 -- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7b97d50 msgr2=0x556ac7b99c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7b97d50 0x556ac7b99c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 -- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 msgr2=0x556ac7bc55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 0x556ac7bc55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 -- 172.21.15.49:0/3185215250 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac78132d0 con 0x556ac7bc51c0 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc0137fe640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 0x556ac7bc55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc00402a770 tx=0x7fc00402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fa8e48e93dd6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc008ff9640 1 -- 172.21.15.49:0/3185215250 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00404e070 con 0x556ac7bc51c0 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc008ff9640 1 -- 172.21.15.49:0/3185215250 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc004032da0 con 0x556ac7bc51c0 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.234+0000 7fc008ff9640 1 -- 172.21.15.49:0/3185215250 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004037040 con 0x556ac7bc51c0 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- 172.21.15.49:0/3185215250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 msgr2=0x556ac7be0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc00402a770 tx=0x7fc00402c000 comp rx=0 tx=0).stop 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- 172.21.15.49:0/3185215250 shutdown_connections 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7b9a1a0 0x556ac7bc55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7b97d50 0x556ac7b99c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3185215250 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bc51c0 0x556ac7be0a60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- 172.21.15.49:0/3185215250 >> 172.21.15.49:0/3185215250 conn(0x556ac7b98be0 msgr2=0x556ac7be00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- 172.21.15.49:0/3185215250 shutdown_connections 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- 172.21.15.49:0/3185215250 wait complete. 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 -- start start 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.235+0000 7fc01c40e740 1 Processor -- start 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bc51c0 0x556ac7bd5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bdf350 0x556ac7bd83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ac7b981e0 con 0x556ac7bed070 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ac7aba8e0 con 0x556ac7bc51c0 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x556ac7b9e2c0 con 0x556ac7bdf350 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc01c40e740 10 client.0 _ll_register_callbacks cb 0x556ac7a879c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34298/0 (socket says 172.21.15.49:34298) 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc0137fe640 1 -- 172.21.15.49:0/1986690320 learned_addr learned my addr 172.21.15.49:0/1986690320 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc012ffd640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bc51c0 0x556ac7bd5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.236+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bdf350 0x556ac7bd83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 -- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bdf350 msgr2=0x556ac7bd83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bdf350 0x556ac7bd83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 -- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bc51c0 msgr2=0x556ac7bd5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bc51c0 0x556ac7bd5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 -- 172.21.15.49:0/1986690320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac78dd6b0 con 0x556ac7bed070 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc0137fe640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0620d0 tx=0x7fc00c05a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eeb99a9fff7426 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c05ddd0 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc00c084cb0 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc004032da0 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc00c062600 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 10 client.4746 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x556ac7b9d7c0 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc004032da0 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fc00c062170 con 0x556ac7bed070 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 10 client.4746 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 10 client.4746 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 20 client.4746 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bf0df0 0x556ac7bfc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 4 client.4746 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:56:36.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.237+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 --> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ac7b0cc00 con 0x556ac7bf0df0 2024-05-09T08:56:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.238+0000 7fc012ffd640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bf0df0 0x556ac7bfc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.238+0000 7fc012ffd640 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bf0df0 0x556ac7bfc5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44a03b93d4cf0020 server_cookie=1d8ff509b60152de in_seq=0 out_seq=0 2024-05-09T08:56:36.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.238+0000 7fbff1ffb640 10 client.4746 ms_handle_connect on v2:172.21.15.49:6833/204294090 2024-05-09T08:56:36.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fbff1ffb640 1 -- 172.21.15.49:0/1986690320 <== mds.1 v2:172.21.15.49:6833/204294090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ac7b0cc00 con 0x556ac7bf0df0 2024-05-09T08:56:36.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fbff1ffb640 10 client.4746 handle_command_reply: tid=0 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 client.4746 shutdown 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 msgr2=0x556ac7bed4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0620d0 tx=0x7fc00c05a120 comp rx=0 tx=0).stop 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 shutdown_connections 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bf0df0 0x556ac7bfc5f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.641+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7bed4f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bdf350 0x556ac7bd83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fbff1ffb640 0 client.4746 ms_handle_reset on v2:172.21.15.49:6833/204294090 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 1 --2- 172.21.15.49:0/1986690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bc51c0 0x556ac7bd5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 >> 172.21.15.49:0/1986690320 conn(0x556ac7b9a1a0 msgr2=0x556ac7bc3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 shutdown_connections 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 1 -- 172.21.15.49:0/1986690320 wait complete. 2024-05-09T08:56:36.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.642+0000 7fc01c40e740 20 client.4746 trim_cache size 0 max 16384 2024-05-09T08:56:36.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 Processor -- start 2024-05-09T08:56:36.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 -- start start 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7bc5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bed070 0x556ac7bd58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141f3e90 con 0x556ac7bdf350 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ac7bd7900 con 0x556ac7bed070 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x556ac7bd8b10 con 0x556ac7bc51c0 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.661+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34310/0 (socket says 172.21.15.49:34310) 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 -- 172.21.15.49:0/2009540881 learned_addr learned my addr 172.21.15.49:0/2009540881 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc012ffd640 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7bc5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc0137fe640 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bed070 0x556ac7bd58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 -- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 msgr2=0x556ac7bc5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7bc5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 -- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bed070 msgr2=0x556ac7bd58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bed070 0x556ac7bd58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 -- 172.21.15.49:0/2009540881 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ac7bdf730 con 0x556ac7bdf350 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0076a0 tx=0x7fbffc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994691c9eb02eb28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fbff3fff640 1 -- 172.21.15.49:0/2009540881 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc019070 con 0x556ac7bdf350 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fbff3fff640 1 -- 172.21.15.49:0/2009540881 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc005e10 con 0x556ac7bdf350 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.662+0000 7fbff3fff640 1 -- 172.21.15.49:0/2009540881 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc012c70 con 0x556ac7bdf350 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 -- 172.21.15.49:0/2009540881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 msgr2=0x556ac7bd5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0076a0 tx=0x7fbffc005bc0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 -- 172.21.15.49:0/2009540881 shutdown_connections 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bed070 0x556ac7bd58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bdf350 0x556ac7bd5360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 --2- 172.21.15.49:0/2009540881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7bc5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 -- 172.21.15.49:0/2009540881 >> 172.21.15.49:0/2009540881 conn(0x556ac7b9a1a0 msgr2=0x556ac7b9b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 -- 172.21.15.49:0/2009540881 shutdown_connections 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.663+0000 7fc01c40e740 1 -- 172.21.15.49:0/2009540881 wait complete. 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 -- start start 2024-05-09T08:56:36.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 Processor -- start 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7c01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bdf350 0x556ac7c5b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x556ac7bf0c70 con 0x556ac7bed070 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ac7bf0620 con 0x556ac7bdf350 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x556ac7bf12c0 con 0x556ac7bc51c0 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.664+0000 7fc01c40e740 10 client.0 _ll_register_callbacks cb 0x7fc00c007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34322/0 (socket says 172.21.15.49:34322) 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 -- 172.21.15.49:0/3259914738 learned_addr learned my addr 172.21.15.49:0/3259914738 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc00a7fc640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bdf350 0x556ac7c5b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7c01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 -- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 msgr2=0x556ac7c01ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7c01ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 -- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bdf350 msgr2=0x556ac7c5b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bdf350 0x556ac7c5b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 -- 172.21.15.49:0/3259914738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ac7c5bc30 con 0x556ac7bed070 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc0137fe640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08ba90 tx=0x7fc00c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8abcdc64bf9e017 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fbff1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c07d060 con 0x556ac7bed070 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc00c059070 con 0x556ac7bed070 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbffc005e10 con 0x556ac7bed070 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc00c083050 con 0x556ac7bed070 2024-05-09T08:56:36.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc01c40e740 10 client.4752 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.665+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x556ac7b95810 con 0x556ac7bed070 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc005e10 con 0x556ac7bed070 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fc00c0831f0 con 0x556ac7bed070 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 10 client.4752 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 10 client.4752 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 20 client.4752 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bd9810 0x556ac7c64440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 4 client.4752 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 --> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] -- command(tid 0: {"prefix": "status"}) -- 0x556ac7b06f50 con 0x556ac7bd9810 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bd9810 0x556ac7c64440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fbff1ffb640 10 client.4752 ms_handle_connect on v2:172.21.15.49:6833/204294090 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc012ffd640 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bd9810 0x556ac7c64440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e73959ef2fc0b09 server_cookie=2c07667f03974e2c in_seq=0 out_seq=0 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fbff1ffb640 1 -- 172.21.15.49:0/3259914738 <== mds.1 v2:172.21.15.49:6833/204294090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556ac7b06f50 con 0x556ac7bd9810 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fbff1ffb640 10 client.4752 handle_command_reply: tid=0 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.666+0000 7fc01c40e740 1 client.4752 shutdown 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 msgr2=0x556ac7c51500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc00c08ba90 tx=0x7fc00c020040 comp rx=0 tx=0).stop 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 shutdown_connections 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:6833/204294090,v1:172.21.15.49:6834/204294090] conn(0x556ac7bd9810 0x556ac7c64440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x556ac7bed070 0x556ac7c51500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ac7bdf350 0x556ac7c5b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 --2- 172.21.15.49:0/3259914738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x556ac7bc51c0 0x556ac7c01ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fbff1ffb640 0 client.4752 ms_handle_reset on v2:172.21.15.49:6833/204294090 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 >> 172.21.15.49:0/3259914738 conn(0x556ac7b9a1a0 msgr2=0x556ac7bdf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 shutdown_connections 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 1 -- 172.21.15.49:0/3259914738 wait complete. 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01c40e740 20 client.4752 trim_cache size 0 max 16384 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:36.673 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 1, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4251, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 29.957325311999998, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 16, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 20, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T08:56:36.674 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 41.545524939000003 2024-05-09T08:56:36.675 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:36.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 msgr2=0x7fbff406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 0x7fbff406e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc004034000 tx=0x7fc00402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 msgr2=0x7fc014159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.667+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc00c007e40 tx=0x7fc00c00e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 shutdown_connections 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbff406be30 0x7fbff406e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc01415e4b0 0x7fc014159620 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc014155080 0x7fc0141590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 --2- 172.21.15.49:0/102528936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0140c96f0 0x7fc014155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 >> 172.21.15.49:0/102528936 conn(0x7fc014072230 msgr2=0x7fc01406b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 shutdown_connections 2024-05-09T08:56:36.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.668+0000 7fc01962c640 1 -- 172.21.15.49:0/102528936 wait complete. 2024-05-09T08:56:36.679 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-09T08:56:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.787+0000 7fafe7190640 1 Processor -- start 2024-05-09T08:56:36.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.787+0000 7fafe7190640 1 -- start start 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fafe0151be0 0x7fafe00c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafe00c71d0 0x7fafe006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fafe00c5c70 con 0x7fafe00c71d0 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fafe00c4b10 con 0x7fafe0151be0 2024-05-09T08:56:36.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe7190640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00c4da0 con 0x7fafe015af60 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafe00c71d0 0x7fafe006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fafe00c71d0 0x7fafe006dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34330/0 (socket says 172.21.15.49:34330) 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 -- 172.21.15.49:0/819763309 learned_addr learned my addr 172.21.15.49:0/819763309 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 -- 172.21.15.49:0/819763309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 msgr2=0x7fafe00c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafd7fff640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/819763309 >> v2:172.21.15.183:3300/0 conn(0x7fafe0151be0 0x7fafe00c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe00c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 -- 172.21.15.49:0/819763309 >> v2:172.21.15.183:3300/0 conn(0x7fafe0151be0 msgr2=0x7fafe00c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 --2- 172.21.15.49:0/819763309 >> v2:172.21.15.183:3300/0 conn(0x7fafe0151be0 0x7fafe00c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.788+0000 7fafe572d640 1 -- 172.21.15.49:0/819763309 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00c5060 con 0x7fafe00c71d0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.789+0000 7fafe572d640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe006dac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fafe006d1a0 tx=0x7fafd802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ad8a64affe7ad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.789+0000 7fafd77fe640 1 -- 172.21.15.49:0/819763309 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8036070 con 0x7fafe00c71d0 2024-05-09T08:56:36.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.789+0000 7fafd77fe640 1 -- 172.21.15.49:0/819763309 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8031600 con 0x7fafe00c71d0 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.789+0000 7fafd77fe640 1 -- 172.21.15.49:0/819763309 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80319d0 con 0x7fafe00c71d0 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 -- 172.21.15.49:0/819763309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 msgr2=0x7fafe006dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe006dac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fafe006d1a0 tx=0x7fafd802a730 comp rx=0 tx=0).stop 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 -- 172.21.15.49:0/819763309 shutdown_connections 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe006dac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 --2- 172.21.15.49:0/819763309 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe00c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 --2- 172.21.15.49:0/819763309 >> v2:172.21.15.183:3300/0 conn(0x7fafe0151be0 0x7fafe00c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 -- 172.21.15.49:0/819763309 >> 172.21.15.49:0/819763309 conn(0x7fafe00cc290 msgr2=0x7fafe00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 -- 172.21.15.49:0/819763309 shutdown_connections 2024-05-09T08:56:36.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.790+0000 7fafe7190640 1 -- 172.21.15.49:0/819763309 wait complete. 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.791+0000 7fafe7190640 1 Processor -- start 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.791+0000 7fafe7190640 1 -- start start 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 0x7fafe01ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe01eeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00c5900 con 0x7fafe00c71d0 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00c5620 con 0x7fafe0151be0 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fafe0151fc0 con 0x7fafe015af60 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34344/0 (socket says 172.21.15.49:34344) 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 -- 172.21.15.49:0/1475277576 learned_addr learned my addr 172.21.15.49:0/1475277576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafd7fff640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 0x7fafe01ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe572d640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe01eeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 -- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 msgr2=0x7fafe01eeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe01eeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 -- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 msgr2=0x7fafe01ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 0x7fafe01ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 -- 172.21.15.49:0/1475277576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe01f1b30 con 0x7fafe00c71d0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafd7fff640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 0x7fafe01ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c820 tx=0x7fafdc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafd57fa640 1 -- 172.21.15.49:0/1475277576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc015070 con 0x7fafe00c71d0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafd57fa640 1 -- 172.21.15.49:0/1475277576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc004240 con 0x7fafe00c71d0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe01f1e10 con 0x7fafe00c71d0 2024-05-09T08:56:36.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.792+0000 7fafd57fa640 1 -- 172.21.15.49:0/1475277576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc00ce10 con 0x7fafe00c71d0 2024-05-09T08:56:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.793+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe01f23d0 con 0x7fafe00c71d0 2024-05-09T08:56:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.794+0000 7fafd57fa640 1 -- 172.21.15.49:0/1475277576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafdc007830 con 0x7fafe00c71d0 2024-05-09T08:56:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.795+0000 7fafd57fa640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.795+0000 7fafd57fa640 1 -- 172.21.15.49:0/1475277576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fafdc08ad60 con 0x7fafe00c71d0 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.795+0000 7fafd7fff640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.795+0000 7fafe9f7a740 1 Processor -- start 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.795+0000 7fafe9f7a740 1 -- start start 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 0x7fafcc06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafe0150eb0 tx=0x7fafd0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e300 0x5572a734b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a739dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572a72b34d0 con 0x5572a734f7f0 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572a725b3e0 con 0x5572a734e6e0 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x5572a72af490 con 0x5572a734e300 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34346/0 (socket says 172.21.15.49:34346) 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 -- 172.21.15.49:0/3795848696 learned_addr learned my addr 172.21.15.49:0/3795848696 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e300 0x5572a734b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafe572d640 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a739dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 -- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e300 msgr2=0x5572a734b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e300 0x5572a734b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 -- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 msgr2=0x5572a739dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a739dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.796+0000 7fafd7fff640 1 -- 172.21.15.49:0/3795848696 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a6fc72d0 con 0x5572a734f7f0 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fafd000a340 tx=0x7fafd0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e91fc8c0a980d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafc67fc640 1 -- 172.21.15.49:0/3795848696 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0026070 con 0x5572a734f7f0 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafc67fc640 1 -- 172.21.15.49:0/3795848696 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0003610 con 0x5572a734f7f0 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafc67fc640 1 -- 172.21.15.49:0/3795848696 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00039c0 con 0x5572a734f7f0 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3795848696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 msgr2=0x5572a7351f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fafd000a340 tx=0x7fafd0012000 comp rx=0 tx=0).stop 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3795848696 shutdown_connections 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a739dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7351f70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3795848696 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e300 0x5572a734b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3795848696 >> 172.21.15.49:0/3795848696 conn(0x5572a734cbe0 msgr2=0x5572a73940b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3795848696 shutdown_connections 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.797+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3795848696 wait complete. 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.798+0000 7fafe9f7a740 1 -- start start 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.798+0000 7fafe9f7a740 1 Processor -- start 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e6e0 0x5572a73932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734f7f0 0x5572a7388040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572a734c400 con 0x5572a7398090 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572a726e8e0 con 0x5572a734f7f0 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x5572a7351710 con 0x5572a734e6e0 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe9f7a740 10 client.0 _ll_register_callbacks cb 0x5572a723b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34362/0 (socket says 172.21.15.49:34362) 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 -- 172.21.15.49:0/3506879859 learned_addr learned my addr 172.21.15.49:0/3506879859 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734f7f0 0x5572a7388040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e6e0 0x5572a73932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 -- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e6e0 msgr2=0x5572a73932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e6e0 0x5572a73932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 -- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734f7f0 msgr2=0x5572a7388040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734f7f0 0x5572a7388040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 -- 172.21.15.49:0/3506879859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a70916b0 con 0x5572a7398090 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafe572d640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fafd802ad20 tx=0x7fafd8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25766144682c38d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.799+0000 7fafbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8036040 con 0x5572a7398090 2024-05-09T08:56:36.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fafd8046b30 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd0003610 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fafd8046d30 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 10 client.4764 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5572a73973c0 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0003610 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fafd8031b20 con 0x5572a7398090 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 10 client.4764 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 10 client.4764 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 20 client.4764 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a73a8f80 0x5572a73b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 4 client.4764 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 --> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572a72c0c00 con 0x5572a73a8f80 2024-05-09T08:56:36.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a73a8f80 0x5572a73b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafbf7fe640 10 client.4764 ms_handle_connect on v2:172.21.15.49:6832/2800676488 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.800+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a73a8f80 0x5572a73b4110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1cbd0dc10cead57 server_cookie=751dacf90a192b5a in_seq=0 out_seq=0 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3506879859 <== mds.0 v2:172.21.15.49:6832/2800676488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572a72c0c00 con 0x5572a73a8f80 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafbf7fe640 10 client.4764 handle_command_reply: tid=0 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 client.4764 shutdown 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 msgr2=0x5572a73a87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fafd802ad20 tx=0x7fafd8004ab0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 shutdown_connections 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a73a8f80 0x5572a73b4110 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a7398090 0x5572a73a87f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734f7f0 0x5572a7388040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafbf7fe640 0 client.4764 ms_handle_reset on v2:172.21.15.49:6832/2800676488 2024-05-09T08:56:36.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3506879859 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734e6e0 0x5572a73932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 >> 172.21.15.49:0/3506879859 conn(0x5572a734e300 msgr2=0x5572a7379cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.801+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 shutdown_connections 2024-05-09T08:56:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.802+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3506879859 wait complete. 2024-05-09T08:56:36.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.802+0000 7fafe9f7a740 20 client.4764 trim_cache size 0 max 16384 2024-05-09T08:56:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 Processor -- start 2024-05-09T08:56:36.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 -- start start 2024-05-09T08:56:36.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a7387000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 0x5572a73a82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fafe0163600 con 0x5572a734f7f0 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572a73976a0 con 0x5572a734e6e0 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.820+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x5572a7395bc0 con 0x5572a7398090 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34366/0 (socket says 172.21.15.49:34366) 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 -- 172.21.15.49:0/3903064007 learned_addr learned my addr 172.21.15.49:0/3903064007 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a7387000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafe572d640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 0x5572a73a82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 -- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 msgr2=0x5572a73a82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 0x5572a73a82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 -- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 msgr2=0x5572a7387000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a7387000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 -- 172.21.15.49:0/3903064007 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a738ab00 con 0x5572a734f7f0 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafe572d640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 0x5572a73a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a7387000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fafd0020000 tx=0x7fafd001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e1f6908080f328 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafc5ffb640 1 -- 172.21.15.49:0/3903064007 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0031070 con 0x5572a734f7f0 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafc5ffb640 1 -- 172.21.15.49:0/3903064007 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0005b90 con 0x5572a734f7f0 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.821+0000 7fafc5ffb640 1 -- 172.21.15.49:0/3903064007 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0026040 con 0x5572a734f7f0 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3903064007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 msgr2=0x5572a7387540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fafd0020000 tx=0x7fafd001e000 comp rx=0 tx=0).stop 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3903064007 shutdown_connections 2024-05-09T08:56:36.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a7398090 0x5572a73a82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734f7f0 0x5572a7387540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3903064007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a734e6e0 0x5572a7387000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3903064007 >> 172.21.15.49:0/3903064007 conn(0x5572a734e300 msgr2=0x5572a73797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3903064007 shutdown_connections 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3903064007 wait complete. 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.822+0000 7fafe9f7a740 1 -- start start 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 Processor -- start 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 0x5572a7412f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 0x5572a74093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x5572a73a9450 con 0x5572a734e6e0 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572a73a8e00 con 0x5572a7398090 2024-05-09T08:56:36.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x5572a7378a80 con 0x5572a734f7f0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe9f7a740 10 client.0 _ll_register_callbacks cb 0x7fafd8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe4f2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe4f2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34374/0 (socket says 172.21.15.49:34374) 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe4f2c640 1 -- 172.21.15.49:0/3295638222 learned_addr learned my addr 172.21.15.49:0/3295638222 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.823+0000 7fafe572d640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 0x5572a74093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 0x5572a7412f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 -- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 msgr2=0x5572a7412f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 0x5572a7412f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 -- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 msgr2=0x5572a74093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 0x5572a74093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 -- 172.21.15.49:0/3295638222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a7391ee0 con 0x5572a734e6e0 2024-05-09T08:56:36.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 0x5572a7412f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe572d640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 0x5572a74093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe4f2c640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fafdc058f10 tx=0x7fafdc01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f942abbd2b4ed2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.49:3300/0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc01e040 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mon.0 v2:172.21.15.49:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fafdc057280 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd0005b90 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fafdc07e050 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 10 client.4770 fetch_fsmap learned FSMap version 16 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5572a734c1d0 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mon.0 v2:172.21.15.49:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0005b90 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mon.0 v2:172.21.15.49:3300/0 5 ==== fsmap(e 16) ==== 4784+0+0 (secure 0 0 0) 0x7fafdc081080 con 0x5572a734e6e0 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 10 client.4770 fetch_fsmap finished waiting for FSMap version 16 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 10 client.4770 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-09T08:56:36.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 20 client.4770 populate_metadata read hostname 'smithi049' 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a7350750 0x5572a741c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 4 client.4770 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.824+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 --> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] -- command(tid 0: {"prefix": "status"}) -- 0x5572a7389a60 con 0x5572a7350750 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a7350750 0x5572a741c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafbf7fe640 10 client.4770 ms_handle_connect on v2:172.21.15.49:6832/2800676488 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafd7fff640 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a7350750 0x5572a741c250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6da146dd37e34b94 server_cookie=86a76b7c1867db82 in_seq=0 out_seq=0 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafbf7fe640 1 -- 172.21.15.49:0/3295638222 <== mds.0 v2:172.21.15.49:6832/2800676488 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5572a7389a60 con 0x5572a7350750 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafbf7fe640 10 client.4770 handle_command_reply: tid=0 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 client.4770 shutdown 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 msgr2=0x5572a7379890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fafdc058f10 tx=0x7fafdc01b040 comp rx=0 tx=0).stop 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 shutdown_connections 2024-05-09T08:56:36.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:6832/2800676488,v1:172.21.15.49:6835/2800676488] conn(0x5572a7350750 0x5572a741c250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572a7398090 0x5572a74093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x5572a734f7f0 0x5572a7412f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafbf7fe640 0 client.4770 ms_handle_reset on v2:172.21.15.49:6832/2800676488 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 --2- 172.21.15.49:0/3295638222 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x5572a734e6e0 0x5572a7379890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 >> 172.21.15.49:0/3295638222 conn(0x5572a734e300 msgr2=0x5572a73a82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 shutdown_connections 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.825+0000 7fafe9f7a740 1 -- 172.21.15.49:0/3295638222 wait complete. 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe9f7a740 20 client.4770 trim_cache size 0 max 16384 2024-05-09T08:56:36.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "whoami": 0, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "id": 4285, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "want_state": "up:active", 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "state": "up:active", 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "fs_name": "cephfs", 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "rank_uptime": 35.150387711999997, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "mdsmap_epoch": 16, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch": 20, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "uptime": 41.704123654999997 2024-05-09T08:56:36.835 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 msgr2=0x7fafcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 0x7fafcc06e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafe0150eb0 tx=0x7fafd0008040 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 msgr2=0x7fafe0150220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c820 tx=0x7fafdc00cce0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 shutdown_connections 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fafcc06be60 0x7fafcc06e320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fafe015af60 0x7fafe01eeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fafe0151be0 0x7fafe01ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 --2- 172.21.15.49:0/1475277576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fafe00c71d0 0x7fafe0150220 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 >> 172.21.15.49:0/1475277576 conn(0x7fafe00cc290 msgr2=0x7fafe00ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:36.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 shutdown_connections 2024-05-09T08:56:36.838 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:36.826+0000 7fafe7190640 1 -- 172.21.15.49:0/1475277576 wait complete. 2024-05-09T08:56:36.838 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T08:56:36.847 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T08:56:36.847 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T08:56:36.848 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T08:56:36.848 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T08:56:36.879 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T08:56:36.879 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T08:56:36.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:36.879 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-09T08:56:36.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:36.897 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-09T08:56:36.965 INFO:teuthology.orchestra.run.smithi049.stderr:Cannot find device "ceph-brx" 2024-05-09T08:56:36.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:56:36.966 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-09T08:56:36.967 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T08:56:36.967 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:36.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T08:56:37.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.111+0000 7f53af5ab640 1 Processor -- start 2024-05-09T08:56:37.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.111+0000 7f53af5ab640 1 -- start start 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53af5ab640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8071bc0 0x7f53a80ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53af5ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80ca7a0 0x7f53a815b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53af5ab640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a806f900 con 0x7f53a8154f10 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53af5ab640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f53a8072000 con 0x7f53a8071bc0 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53af5ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53a815e340 con 0x7f53a80ca7a0 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8071bc0 0x7f53a80ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8071bc0 0x7f53a80ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34382/0 (socket says 172.21.15.49:34382) 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 -- 172.21.15.49:0/3439798050 learned_addr learned my addr 172.21.15.49:0/3439798050 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53adb48640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80ca7a0 0x7f53a815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 -- 172.21.15.49:0/3439798050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80ca7a0 msgr2=0x7f53a815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80ca7a0 0x7f53a815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 -- 172.21.15.49:0/3439798050 >> v1:172.21.15.183:6789/0 conn(0x7f53a8154f10 legacy=0x7f53a806f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.112+0000 7f53acb46640 1 -- 172.21.15.49:0/3439798050 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a815e620 con 0x7f53a8071bc0 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.113+0000 7f53acb46640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a80ca260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f539c009870 tx=0x7f539c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fe2bf90bfcdf07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.113+0000 7f53a7fff640 1 -- 172.21.15.49:0/3439798050 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c037070 con 0x7f53a8071bc0 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.113+0000 7f53a7fff640 1 -- 172.21.15.49:0/3439798050 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539c032440 con 0x7f53a8071bc0 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.113+0000 7f53a7fff640 1 -- 172.21.15.49:0/3439798050 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c032810 con 0x7f53a8071bc0 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 -- 172.21.15.49:0/3439798050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 msgr2=0x7f53a80ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a80ca260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f539c009870 tx=0x7f539c02a270 comp rx=0 tx=0).stop 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 -- 172.21.15.49:0/3439798050 shutdown_connections 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80ca7a0 0x7f53a815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3439798050 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a80ca260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 -- 172.21.15.49:0/3439798050 >> 172.21.15.49:0/3439798050 conn(0x7f53a806b3e0 msgr2=0x7f53a806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:37.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 -- 172.21.15.49:0/3439798050 shutdown_connections 2024-05-09T08:56:37.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.114+0000 7f53af5ab640 1 -- 172.21.15.49:0/3439798050 wait complete. 2024-05-09T08:56:37.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.115+0000 7f53af5ab640 1 Processor -- start 2024-05-09T08:56:37.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.115+0000 7f53af5ab640 1 -- start start 2024-05-09T08:56:37.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80ca7a0 0x7f53a8146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a8154f10 0x7f53a81472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53a815eec0 con 0x7f53a8071bc0 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53a815ebe0 con 0x7f53a80ca7a0 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53af5ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53a815f1a0 con 0x7f53a8154f10 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34390/0 (socket says 172.21.15.49:34390) 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 -- 172.21.15.49:0/3937788642 learned_addr learned my addr 172.21.15.49:0/3937788642 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53adb48640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a8154f10 0x7f53a81472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53acb46640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80ca7a0 0x7f53a8146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 -- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a8154f10 msgr2=0x7f53a81472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a8154f10 0x7f53a81472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 -- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80ca7a0 msgr2=0x7f53a8146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80ca7a0 0x7f53a8146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 -- 172.21.15.49:0/3937788642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a8157bb0 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.116+0000 7f53ad347640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c3c0 tx=0x7f53a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.117+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a00052d0 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.117+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a0005490 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.117+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a8157e90 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.117+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0007260 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.117+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a80cb280 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.118+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a80c8120 con 0x7f53a8071bc0 2024-05-09T08:56:37.119 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.118+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53a0007400 con 0x7f53a8071bc0 2024-05-09T08:56:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.119+0000 7f53a5ffb640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.119+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53a008a3f0 con 0x7f53a8071bc0 2024-05-09T08:56:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.119+0000 7f53acb46640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:37.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.120+0000 7f53acb46640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:37.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.123+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f53a80c8120 con 0x7f53a8071bc0 2024-05-09T08:56:37.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.344+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53a8068560 con 0x7f53a8071bc0 2024-05-09T08:56:37.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.344+0000 7f53a5ffb640 1 -- 172.21.15.49:0/3937788642 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f53a0058d10 con 0x7f53a8071bc0 2024-05-09T08:56:37.344 INFO:teuthology.orchestra.run.smithi049.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 msgr2=0x7f539406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f539c034000 tx=0x7f539c02e000 comp rx=0 tx=0).stop 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 msgr2=0x7f53a815b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c3c0 tx=0x7f53a000c880 comp rx=0 tx=0).stop 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 shutdown_connections 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a8154f10 0x7f53a81472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80ca7a0 0x7f53a8146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 --2- 172.21.15.49:0/3937788642 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8071bc0 0x7f53a815b0c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 >> 172.21.15.49:0/3937788642 conn(0x7f53a806b3e0 msgr2=0x7f53a8153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 shutdown_connections 2024-05-09T08:56:37.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37.346+0000 7f53af5ab640 1 -- 172.21.15.49:0/3937788642 wait complete. 2024-05-09T08:56:37.356 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi049.front.sepia.ceph.com 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T08:56:37.357 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:56:37.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.358 DEBUG:teuthology.orchestra.run.smithi049:> ip addr 2024-05-09T08:56:37.374 INFO:teuthology.orchestra.run.smithi049.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 ::1/128 scope host 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:84 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f0 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:6b:ff:85 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:56:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp7s0f1 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:66 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f0 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: inet 172.21.15.49/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft 42632sec preferred_lft 42632sec 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: inet6 fe80::ec4:7aff:febd:1566/64 scope link noprefixroute 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: valid_lft forever preferred_lft forever 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: link/ether 0c:c4:7a:bd:15:67 brd ff:ff:ff:ff:ff:ff 2024-05-09T08:56:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: altname enp1s0f1 2024-05-09T08:56:37.377 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add name ceph-brx type bridge 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr flush dev ceph-brx 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set ceph-brx up 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T08:56:37.377 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-09T08:56:37.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:37.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:37.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.572 DEBUG:teuthology.orchestra.run.smithi049:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T08:56:37.597 INFO:teuthology.orchestra.run.smithi049.stdout:1 2024-05-09T08:56:37.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.599 DEBUG:teuthology.orchestra.run.smithi049:> route 2024-05-09T08:56:37.672 INFO:teuthology.orchestra.run.smithi049.stdout:Kernel IP routing table 2024-05-09T08:56:37.672 INFO:teuthology.orchestra.run.smithi049.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T08:56:37.672 INFO:teuthology.orchestra.run.smithi049.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T08:56:37.672 INFO:teuthology.orchestra.run.smithi049.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T08:56:37.672 INFO:teuthology.orchestra.run.smithi049.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T08:56:37.673 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-09T08:56:37.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:37.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:37.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.943 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-09T08:56:37.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:37.960 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list-id 2024-05-09T08:56:38.015 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:56:38.015 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-09T08:56:38.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:56:38.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T08:56:38.015 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-09T08:56:38.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:38.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:38.135 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T08:56:38.135 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T08:56:38.136 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-09T08:56:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:38.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:38.269 DEBUG:teuthology.orchestra.run.smithi049:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T08:56:38.269 DEBUG:teuthology.orchestra.run.smithi049:> set -e 2024-05-09T08:56:38.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set brx.0 up 2024-05-09T08:56:38.269 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T08:56:38.269 DEBUG:teuthology.orchestra.run.smithi049:> ') 2024-05-09T08:56:38.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:38.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:38.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:56:38.348 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:38.367 INFO:teuthology.orchestra.run.smithi049.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T08:56:38.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T08:56:38.368 DEBUG:teuthology.orchestra.run.smithi049:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:38.425 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-09T08:56:38.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:38.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T08:56:38.530 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T08:56:38.530 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T08:56:38.642 INFO:teuthology.orchestra.run.smithi049.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T08:56:38.642 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T08:56:38.642 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: resolved to: "172.21.15.49:3300,172.21.15.183:3300,172.21.15.183:3301" 2024-05-09T08:56:38.642 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: trying mount with new device syntax: 0@aab8b2de-6010-439c-b8fd-8bc72445265c.cephfs=/ 2024-05-09T08:56:38.642 INFO:teuthology.orchestra.run.smithi049.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.49:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2024-05-09T08:56:38.645 INFO:teuthology.orchestra.run.smithi049.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T08:56:38.645 INFO:teuthology.orchestra.run.smithi049.stdout: You just mounted a file system that supports labels which does not 2024-05-09T08:56:38.645 INFO:teuthology.orchestra.run.smithi049.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T08:56:38.645 INFO:teuthology.orchestra.run.smithi049.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T08:56:38.645 INFO:teuthology.orchestra.run.smithi049.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T08:56:38.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T08:56:38.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:56:38.646 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:38.682 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:56:38.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:56:38.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:38.786 INFO:teuthology.orchestra.run.smithi049.stdout:client4779 2024-05-09T08:56:38.786 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:56:38.786 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:56:38.803 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:56:39.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:56:39.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:39.809 INFO:teuthology.orchestra.run.smithi049.stdout:client4779 2024-05-09T08:56:39.810 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T08:56:39.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T08:56:39.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:39.881 INFO:teuthology.orchestra.run.smithi049.stdout:client4779 2024-05-09T08:56:39.882 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/aab8b2de-6010-439c-b8fd-8bc72445265c.client4779/status') 2024-05-09T08:56:39.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T08:56:39.948 INFO:teuthology.orchestra.run.smithi049.stdout:instance: client.4779 (3)192.168.0.1:0/2894013287 2024-05-09T08:56:39.949 INFO:teuthology.orchestra.run.smithi049.stdout:blocklisted: false 2024-05-09T08:56:39.949 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records in 2024-05-09T08:56:39.949 INFO:teuthology.orchestra.run.smithi049.stderr:0+1 records out 2024-05-09T08:56:39.949 INFO:teuthology.orchestra.run.smithi049.stderr:69 bytes copied, 2.4556e-05 s, 2.8 MB/s 2024-05-09T08:56:39.949 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T08:56:39.956 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-09T08:56:39.966 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-09T08:56:39.967 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}} 2024-05-09T08:56:39.967 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-09T08:56:39.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-09T08:56:40.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.074+0000 7ff3e044d640 1 Processor -- start 2024-05-09T08:56:40.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.074+0000 7ff3e044d640 1 -- start start 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3e044d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff3d8152e30 0x7ff3d8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3e044d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c7130 0x7ff3d80c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3e044d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d8159b10 con 0x7ff3d80c6450 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3e044d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d8159270 con 0x7ff3d8152e30 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3e044d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d8159550 con 0x7ff3d80c7130 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3de1e9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff3d8152e30 0x7ff3d8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.075+0000 7ff3dd9e8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c7130 0x7ff3d80c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff3d8152e30 0x7ff3d8158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:38144/0 (socket says 172.21.15.49:38144) 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 -- 172.21.15.49:0/2417087724 learned_addr learned my addr 172.21.15.49:0/2417087724 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:40.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de9ea640 1 --1- 172.21.15.49:0/2417087724 >> v1:172.21.15.183:6789/0 conn(0x7ff3d80c6450 0x7ff3d80c3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:53148/0 (socket says 172.21.15.49:53148) 2024-05-09T08:56:40.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 -- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c7130 msgr2=0x7ff3d80c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 --2- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c7130 0x7ff3d80c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 -- 172.21.15.49:0/2417087724 >> v1:172.21.15.183:6789/0 conn(0x7ff3d80c6450 legacy=0x7ff3d80c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.076+0000 7ff3de1e9640 1 -- 172.21.15.49:0/2417087724 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8159830 con 0x7ff3d8152e30 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3de1e9640 1 --2- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8158be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009e50 tx=0x7ff3d402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79db4c9c71d7085 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3dd1e7640 1 -- 172.21.15.49:0/2417087724 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4036070 con 0x7ff3d8152e30 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3dd1e7640 1 -- 172.21.15.49:0/2417087724 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d4031600 con 0x7ff3d8152e30 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3dd1e7640 1 -- 172.21.15.49:0/2417087724 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40319d0 con 0x7ff3d8152e30 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 -- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 msgr2=0x7ff3d8158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 --2- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8158be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009e50 tx=0x7ff3d402a710 comp rx=0 tx=0).stop 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 -- 172.21.15.49:0/2417087724 shutdown_connections 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 --2- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c7130 0x7ff3d80c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 --2- 172.21.15.49:0/2417087724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8158be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.077+0000 7ff3e044d640 1 -- 172.21.15.49:0/2417087724 >> 172.21.15.49:0/2417087724 conn(0x7ff3d80cc290 msgr2=0x7ff3d80cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.078+0000 7ff3e044d640 1 -- 172.21.15.49:0/2417087724 shutdown_connections 2024-05-09T08:56:40.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.078+0000 7ff3e044d640 1 -- 172.21.15.49:0/2417087724 wait complete. 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 Processor -- start 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 -- start start 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c6450 0x7ff3d8148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d806b4b0 con 0x7ff3d80c7130 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d80c59f0 con 0x7ff3d8152e30 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3e044d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d8153210 con 0x7ff3d80c6450 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.079+0000 7ff3dd9e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34406/0 (socket says 172.21.15.49:34406) 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 -- 172.21.15.49:0/4199337498 learned_addr learned my addr 172.21.15.49:0/4199337498 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3de1e9640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c6450 0x7ff3d8148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3de9ea640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 -- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c6450 msgr2=0x7ff3d8148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c6450 0x7ff3d8148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 -- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 msgr2=0x7ff3d8150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 -- 172.21.15.49:0/4199337498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8147c10 con 0x7ff3d80c7130 2024-05-09T08:56:40.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3de9ea640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3dd9e8640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800e9e0 tx=0x7ff3c800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8017070 con 0x7ff3d80c7130 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d81f19f0 con 0x7ff3d80c7130 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c800c9d0 con 0x7ff3d80c7130 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800cdc0 con 0x7ff3d80c7130 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.080+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d8147930 con 0x7ff3d80c7130 2024-05-09T08:56:40.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.081+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d804f660 con 0x7ff3d80c7130 2024-05-09T08:56:40.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.082+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3c801a070 con 0x7ff3d80c7130 2024-05-09T08:56:40.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.082+0000 7ff3ceffd640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 0x7ff3bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.082+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff3c808aea0 con 0x7ff3d80c7130 2024-05-09T08:56:40.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.082+0000 7ff3de1e9640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 0x7ff3bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.083+0000 7ff3de1e9640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 0x7ff3bc06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4033000 tx=0x7ff3d4009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.087+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3c8059180 con 0x7ff3d80c7130 2024-05-09T08:56:40.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.269+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7ff3d8072350 con 0x7ff3d80c7130 2024-05-09T08:56:40.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.272+0000 7ff3ceffd640 1 -- 172.21.15.49:0/4199337498 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7ff3c805d480 con 0x7ff3d80c7130 2024-05-09T08:56:40.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 msgr2=0x7ff3bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 0x7ff3bc06e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4033000 tx=0x7ff3d4009000 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 msgr2=0x7ff3d81eedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800e9e0 tx=0x7ff3c800eea0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 shutdown_connections 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff3bc06be30 0x7ff3bc06e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3d8152e30 0x7ff3d8150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff3d80c7130 0x7ff3d81eedd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 --2- 172.21.15.49:0/4199337498 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff3d80c6450 0x7ff3d8148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 >> 172.21.15.49:0/4199337498 conn(0x7ff3d80cc290 msgr2=0x7ff3d806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 shutdown_connections 2024-05-09T08:56:40.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.274+0000 7ff3e044d640 1 -- 172.21.15.49:0/4199337498 wait complete. 2024-05-09T08:56:40.284 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-09T08:56:40.284 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T08:56:40.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:40.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.388+0000 7fcef24f5640 1 Processor -- start 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fcef24f5640 1 -- start start 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fcef24f5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fceec0cc8a0 0x7fceec15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fcef24f5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec15e260 con 0x7fceec1517f0 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fcef24f5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceec15e520 con 0x7fceec075200 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fcef24f5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fceec151bd0 con 0x7fceec0cc8a0 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fceebfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fceec1517f0 0x7fceec0ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35224/0 (socket says 172.21.15.49:35224) 2024-05-09T08:56:40.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.389+0000 7fceebfff640 1 -- 172.21.15.49:0/1490370700 learned_addr learned my addr 172.21.15.49:0/1490370700 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:40.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428425008 0 0) 0x7fceec15e260 con 0x7fceec1517f0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced4003520 con 0x7fceec1517f0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fcef0a92640 1 --2- 172.21.15.49:0/1490370700 >> v2:172.21.15.183:3300/0 conn(0x7fceec0cc8a0 0x7fceec15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 572227650 0 0) 0x7fced4003520 con 0x7fceec1517f0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 >> v2:172.21.15.183:3300/0 conn(0x7fceec0cc8a0 msgr2=0x7fceec15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 --2- 172.21.15.49:0/1490370700 >> v2:172.21.15.183:3300/0 conn(0x7fceec0cc8a0 0x7fceec15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 >> v1:172.21.15.183:6789/0 conn(0x7fceec075200 legacy=0x7fceec0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec15e800 con 0x7fceec1517f0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fcedc001670 con 0x7fceec1517f0 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 >> v1:172.21.15.49:6789/0 conn(0x7fceec1517f0 legacy=0x7fceec0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 0x7fced4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced4004800 0x7fced4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fced40051c0 0x7fced400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fceec151bd0 con 0x7fced40039b0 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fced400c070 con 0x7fced40051c0 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fced400c270 con 0x7fced4004800 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 0x7fced4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceeb7fe640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced4004800 0x7fced4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fcef0a92640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fced40051c0 0x7fced400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 -- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced4004800 msgr2=0x7fced4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced4004800 0x7fced4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 -- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fced40051c0 msgr2=0x7fced400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fced40051c0 0x7fced400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fcef0a92640 1 -- 172.21.15.49:0/1490370700 reap_dead start 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.390+0000 7fceebfff640 1 -- 172.21.15.49:0/1490370700 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced4003520 con 0x7fced40039b0 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.391+0000 7fceebfff640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 0x7fced4003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c240 tx=0x7fced4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6a1e53f0321d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.391+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007cf0 con 0x7fced40039b0 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.391+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcedc005af0 con 0x7fced40039b0 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.391+0000 7fceeaffd640 1 -- 172.21.15.49:0/1490370700 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc007070 con 0x7fced40039b0 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 -- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 msgr2=0x7fced4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 0x7fced4003dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcedc00c240 tx=0x7fced4005830 comp rx=0 tx=0).stop 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 -- 172.21.15.49:0/1490370700 shutdown_connections 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 --2- 172.21.15.49:0/1490370700 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fced40039b0 0x7fced4003dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 -- 172.21.15.49:0/1490370700 >> 172.21.15.49:0/1490370700 conn(0x7fceec06bd00 msgr2=0x7fceec074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 -- 172.21.15.49:0/1490370700 shutdown_connections 2024-05-09T08:56:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.392+0000 7fcef24f5640 1 -- 172.21.15.49:0/1490370700 wait complete. 2024-05-09T08:56:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.393+0000 7fcef24f5640 1 Processor -- start 2024-05-09T08:56:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.393+0000 7fcef24f5640 1 -- start start 2024-05-09T08:56:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced40039b0 0x7fceec15d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec141cd0 0x7fceec150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fceec15f0a0 con 0x7fceec142710 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fceec15edc0 con 0x7fceec141cd0 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef24f5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fceec0cbc70 con 0x7fced40039b0 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34418/0 (socket says 172.21.15.49:34418) 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 -- 172.21.15.49:0/971363908 learned_addr learned my addr 172.21.15.49:0/971363908 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 -- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced40039b0 msgr2=0x7fceec15d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced40039b0 0x7fceec15d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 -- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec141cd0 msgr2=0x7fceec150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec141cd0 0x7fceec150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.394+0000 7fcef0a92640 1 -- 172.21.15.49:0/971363908 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec150db0 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcef0a92640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fceec0b3200 tx=0x7fcee40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4011de0 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee401b070 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1f1270 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4003990 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.395+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1f1810 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.396+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec068560 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.396+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcee4003b30 con 0x7fceec142710 2024-05-09T08:56:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.397+0000 7fcee8ff9640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 0x7fced806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.397+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcee408b660 con 0x7fceec142710 2024-05-09T08:56:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.397+0000 7fceebfff640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 0x7fced806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.398+0000 7fceebfff640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 0x7fced806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcedc032000 tx=0x7fcedc007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.402+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcee4003e20 con 0x7fceec142710 2024-05-09T08:56:40.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:56:40.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:56:40.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.581+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fceec158620 con 0x7fced806be60 2024-05-09T08:56:40.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.588+0000 7fcee8ff9640 1 -- 172.21.15.49:0/971363908 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7fceec158620 con 0x7fced806be60 2024-05-09T08:56:40.592 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 msgr2=0x7fced806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 0x7fced806e320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcedc032000 tx=0x7fcedc007000 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 msgr2=0x7fceec1597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fceec0b3200 tx=0x7fcee40115a0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 shutdown_connections 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fced806be60 0x7fced806e320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fceec142710 0x7fceec1597c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fceec141cd0 0x7fceec150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 --2- 172.21.15.49:0/971363908 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fced40039b0 0x7fceec15d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 >> 172.21.15.49:0/971363908 conn(0x7fceec06bd00 msgr2=0x7fceec072490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 shutdown_connections 2024-05-09T08:56:40.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.592+0000 7fcef24f5640 1 -- 172.21.15.49:0/971363908 wait complete. 2024-05-09T08:56:40.602 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-09T08:56:40.580606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":363,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235784,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738936,"statfs":{"total":773094113280,"available":772852670464,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.735788+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:11.735788+0000","last_peered":"2024-05-09T08:56:11.735788+0000","last_clean":"2024-05-09T08:56:11.735788+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:11.735788+0000","last_undegraded":"2024-05-09T08:56:11.735788+0000","last_fullsized":"2024-05-09T08:56:11.735788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.752061+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:23.752061+0000","last_peered":"2024-05-09T08:56:23.752061+0000","last_clean":"2024-05-09T08:56:23.752061+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:23.752061+0000","last_undegraded":"2024-05-09T08:56:23.752061+0000","last_fullsized":"2024-05-09T08:56:23.752061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.735501+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:17.735501+0000","last_peered":"2024-05-09T08:56:17.735501+0000","last_clean":"2024-05-09T08:56:17.735501+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:17.735501+0000","last_undegraded":"2024-05-09T08:56:17.735501+0000","last_fullsized":"2024-05-09T08:56:17.735501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.752115+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:23.752115+0000","last_peered":"2024-05-09T08:56:23.752115+0000","last_clean":"2024-05-09T08:56:23.752115+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:23.752115+0000","last_undegraded":"2024-05-09T08:56:23.752115+0000","last_fullsized":"2024-05-09T08:56:23.752115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.733031+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:11.733031+0000","last_peered":"2024-05-09T08:56:11.733031+0000","last_clean":"2024-05-09T08:56:11.733031+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:11.733031+0000","last_undegraded":"2024-05-09T08:56:11.733031+0000","last_fullsized":"2024-05-09T08:56:11.733031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768777+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:29.768777+0000","last_peered":"2024-05-09T08:56:29.768777+0000","last_clean":"2024-05-09T08:56:29.768777+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:29.768777+0000","last_undegraded":"2024-05-09T08:56:29.768777+0000","last_fullsized":"2024-05-09T08:56:29.768777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.731426+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:11.731426+0000","last_peered":"2024-05-09T08:56:11.731426+0000","last_clean":"2024-05-09T08:56:11.731426+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:11.731426+0000","last_undegraded":"2024-05-09T08:56:11.731426+0000","last_fullsized":"2024-05-09T08:56:11.731426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.765633+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:29.765633+0000","last_peered":"2024-05-09T08:56:29.765633+0000","last_clean":"2024-05-09T08:56:29.765633+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:29.765633+0000","last_undegraded":"2024-05-09T08:56:29.765633+0000","last_fullsized":"2024-05-09T08:56:29.765633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.766127+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:29.766127+0000","last_peered":"2024-05-09T08:56:29.766127+0000","last_clean":"2024-05-09T08:56:29.766127+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:29.766127+0000","last_undegraded":"2024-05-09T08:56:29.766127+0000","last_fullsized":"2024-05-09T08:56:29.766127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751628+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:23.751628+0000","last_peered":"2024-05-09T08:56:23.751628+0000","last_clean":"2024-05-09T08:56:23.751628+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:23.751628+0000","last_undegraded":"2024-05-09T08:56:23.751628+0000","last_fullsized":"2024-05-09T08:56:23.751628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751262+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:23.751262+0000","last_peered":"2024-05-09T08:56:23.751262+0000","last_clean":"2024-05-09T08:56:23.751262+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:23.751262+0000","last_undegraded":"2024-05-09T08:56:23.751262+0000","last_fullsized":"2024-05-09T08:56:23.751262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.747643+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:23.747643+0000","last_peered":"2024-05-09T08:56:23.747643+0000","last_clean":"2024-05-09T08:56:23.747643+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:23.747643+0000","last_undegraded":"2024-05-09T08:56:23.747643+0000","last_fullsized":"2024-05-09T08:56:23.747643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686508+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:06.686508+0000","last_peered":"2024-05-09T08:56:06.686508+0000","last_clean":"2024-05-09T08:56:06.686508+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:06.686508+0000","last_undegraded":"2024-05-09T08:56:06.686508+0000","last_fullsized":"2024-05-09T08:56:06.686508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.767451+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:29.767451+0000","last_peered":"2024-05-09T08:56:29.767451+0000","last_clean":"2024-05-09T08:56:29.767451+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:29.767451+0000","last_undegraded":"2024-05-09T08:56:29.767451+0000","last_fullsized":"2024-05-09T08:56:29.767451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750461+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:23.750461+0000","last_peered":"2024-05-09T08:56:23.750461+0000","last_clean":"2024-05-09T08:56:23.750461+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:23.750461+0000","last_undegraded":"2024-05-09T08:56:23.750461+0000","last_fullsized":"2024-05-09T08:56:23.750461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750621+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:23.750621+0000","last_peered":"2024-05-09T08:56:23.750621+0000","last_clean":"2024-05-09T08:56:23.750621+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:23.750621+0000","last_undegraded":"2024-05-09T08:56:23.750621+0000","last_fullsized":"2024-05-09T08:56:23.750621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750728+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:23.750728+0000","last_peered":"2024-05-09T08:56:23.750728+0000","last_clean":"2024-05-09T08:56:23.750728+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:23.750728+0000","last_undegraded":"2024-05-09T08:56:23.750728+0000","last_fullsized":"2024-05-09T08:56:23.750728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.738287+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:17.738287+0000","last_peered":"2024-05-09T08:56:17.738287+0000","last_clean":"2024-05-09T08:56:17.738287+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:17.738287+0000","last_undegraded":"2024-05-09T08:56:17.738287+0000","last_fullsized":"2024-05-09T08:56:17.738287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.735438+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:17.735438+0000","last_peered":"2024-05-09T08:56:17.735438+0000","last_clean":"2024-05-09T08:56:17.735438+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:17.735438+0000","last_undegraded":"2024-05-09T08:56:17.735438+0000","last_fullsized":"2024-05-09T08:56:17.735438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768265+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:29.768265+0000","last_peered":"2024-05-09T08:56:29.768265+0000","last_clean":"2024-05-09T08:56:29.768265+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:29.768265+0000","last_undegraded":"2024-05-09T08:56:29.768265+0000","last_fullsized":"2024-05-09T08:56:29.768265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693296+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693296+0000","last_peered":"2024-05-09T08:56:02.693296+0000","last_clean":"2024-05-09T08:56:02.693296+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693296+0000","last_undegraded":"2024-05-09T08:56:02.693296+0000","last_fullsized":"2024-05-09T08:56:02.693296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.749772+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:23.749772+0000","last_peered":"2024-05-09T08:56:23.749772+0000","last_clean":"2024-05-09T08:56:23.749772+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:23.749772+0000","last_undegraded":"2024-05-09T08:56:23.749772+0000","last_fullsized":"2024-05-09T08:56:23.749772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751566+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:23.751566+0000","last_peered":"2024-05-09T08:56:23.751566+0000","last_clean":"2024-05-09T08:56:23.751566+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:23.751566+0000","last_undegraded":"2024-05-09T08:56:23.751566+0000","last_fullsized":"2024-05-09T08:56:23.751566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768513+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:29.768513+0000","last_peered":"2024-05-09T08:56:29.768513+0000","last_clean":"2024-05-09T08:56:29.768513+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:29.768513+0000","last_undegraded":"2024-05-09T08:56:29.768513+0000","last_fullsized":"2024-05-09T08:56:29.768513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.730452+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:11.730452+0000","last_peered":"2024-05-09T08:56:11.730452+0000","last_clean":"2024-05-09T08:56:11.730452+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:11.730452+0000","last_undegraded":"2024-05-09T08:56:11.730452+0000","last_fullsized":"2024-05-09T08:56:11.730452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.769188+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:29.769188+0000","last_peered":"2024-05-09T08:56:29.769188+0000","last_clean":"2024-05-09T08:56:29.769188+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:29.769188+0000","last_undegraded":"2024-05-09T08:56:29.769188+0000","last_fullsized":"2024-05-09T08:56:29.769188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:24.729364+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:24.729364+0000","last_peered":"2024-05-09T08:56:24.729364+0000","last_clean":"2024-05-09T08:56:24.729364+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:24.729364+0000","last_undegraded":"2024-05-09T08:56:24.729364+0000","last_fullsized":"2024-05-09T08:56:24.729364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.721112+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:11.721112+0000","last_peered":"2024-05-09T08:56:11.721112+0000","last_clean":"2024-05-09T08:56:11.721112+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:11.721112+0000","last_undegraded":"2024-05-09T08:56:11.721112+0000","last_fullsized":"2024-05-09T08:56:11.721112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768412+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:29.768412+0000","last_peered":"2024-05-09T08:56:29.768412+0000","last_clean":"2024-05-09T08:56:29.768412+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:29.768412+0000","last_undegraded":"2024-05-09T08:56:29.768412+0000","last_fullsized":"2024-05-09T08:56:29.768412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'11","reported_seq":25,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768783+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:29.768783+0000","last_peered":"2024-05-09T08:56:29.768783+0000","last_clean":"2024-05-09T08:56:29.768783+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:29.768783+0000","last_undegraded":"2024-05-09T08:56:29.768783+0000","last_fullsized":"2024-05-09T08:56:29.768783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685096+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:06.685096+0000","last_peered":"2024-05-09T08:56:06.685096+0000","last_clean":"2024-05-09T08:56:06.685096+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:06.685096+0000","last_undegraded":"2024-05-09T08:56:06.685096+0000","last_fullsized":"2024-05-09T08:56:06.685096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768497+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:29.768497+0000","last_peered":"2024-05-09T08:56:29.768497+0000","last_clean":"2024-05-09T08:56:29.768497+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:29.768497+0000","last_undegraded":"2024-05-09T08:56:29.768497+0000","last_fullsized":"2024-05-09T08:56:29.768497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:40.607 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:56:40.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.747+0000 7fdddef14640 1 Processor -- start 2024-05-09T08:56:40.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.747+0000 7fdddef14640 1 -- start start 2024-05-09T08:56:40.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddef14640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fddd81517f0 0x7fddd815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddef14640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd815e260 con 0x7fddd80cc190 2024-05-09T08:56:40.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddef14640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd815e520 con 0x7fddd8075200 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddef14640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fddd8151bd0 con 0x7fddd81517f0 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddd4b1640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fddd80cc190 0x7fddd80cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35240/0 (socket says 172.21.15.49:35240) 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.748+0000 7fdddd4b1640 1 -- 172.21.15.49:0/2782751533 learned_addr learned my addr 172.21.15.49:0/2782751533 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fdddccb0640 1 --2- 172.21.15.49:0/2782751533 >> v2:172.21.15.183:3300/0 conn(0x7fddd81517f0 0x7fddd815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239792825 0 0) 0x7fddd815e260 con 0x7fddd80cc190 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4003520 con 0x7fddd80cc190 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869629873 0 0) 0x7fddc4003520 con 0x7fddd80cc190 2024-05-09T08:56:40.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 >> v2:172.21.15.183:3300/0 conn(0x7fddd81517f0 msgr2=0x7fddd815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 --2- 172.21.15.49:0/2782751533 >> v2:172.21.15.183:3300/0 conn(0x7fddd81517f0 0x7fddd815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 >> v1:172.21.15.183:6789/0 conn(0x7fddd8075200 legacy=0x7fddd80ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd815e800 con 0x7fddd80cc190 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fddc0001670 con 0x7fddd80cc190 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 >> v1:172.21.15.49:6789/0 conn(0x7fddd80cc190 legacy=0x7fddd80cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddc40039b0 0x7fddc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddc4004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc4005080 0x7fddc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8151bd0 con 0x7fddc40043b0 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fddc400bf30 con 0x7fddc4005080 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fddc400c130 con 0x7fddc40039b0 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddc4004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fdddd4b1640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc4005080 0x7fddc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fdddccb0640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddc40039b0 0x7fddc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 -- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddc40039b0 msgr2=0x7fddc4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddc40039b0 0x7fddc4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 -- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc4005080 msgr2=0x7fddc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddc4005080 0x7fddc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fddd7fff640 1 -- 172.21.15.49:0/2782751533 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4003520 con 0x7fddc40043b0 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.749+0000 7fdddd4b1640 1 -- 172.21.15.49:0/2782751533 reap_dead start 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.750+0000 7fddd7fff640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddc4004880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fddc8001f60 tx=0x7fddc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e545993ee567f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.750+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8001b30 con 0x7fddc40043b0 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.750+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc8031620 con 0x7fddc40043b0 2024-05-09T08:56:40.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.750+0000 7fddd77fe640 1 -- 172.21.15.49:0/2782751533 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8031a10 con 0x7fddc40043b0 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 -- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 msgr2=0x7fddc4004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddc4004880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fddc8001f60 tx=0x7fddc4005830 comp rx=0 tx=0).stop 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 -- 172.21.15.49:0/2782751533 shutdown_connections 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 --2- 172.21.15.49:0/2782751533 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddc4004880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 -- 172.21.15.49:0/2782751533 >> 172.21.15.49:0/2782751533 conn(0x7fddd806bd00 msgr2=0x7fddd8074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 -- 172.21.15.49:0/2782751533 shutdown_connections 2024-05-09T08:56:40.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.751+0000 7fdddef14640 1 -- 172.21.15.49:0/2782751533 wait complete. 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.752+0000 7fdddef14640 1 Processor -- start 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.752+0000 7fdddef14640 1 -- start start 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddd80c7c70 0x7fddd81e4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd80ca1f0 0x7fddd81e4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fddd815f0a0 con 0x7fddc40043b0 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fddd815edc0 con 0x7fddd80ca1f0 2024-05-09T08:56:40.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddef14640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80cbc70 con 0x7fddd80c7c70 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34432/0 (socket says 172.21.15.49:34432) 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 -- 172.21.15.49:0/3465874910 learned_addr learned my addr 172.21.15.49:0/3465874910 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 -- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddd80c7c70 msgr2=0x7fddd81e4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddd80c7c70 0x7fddd81e4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 -- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd80ca1f0 msgr2=0x7fddd81e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd80ca1f0 0x7fddd81e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 -- 172.21.15.49:0/3465874910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81e65e0 con 0x7fddc40043b0 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.753+0000 7fdddccb0640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fddd0011aa0 tx=0x7fddd0011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.754+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd000fb00 con 0x7fddc40043b0 2024-05-09T08:56:40.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.754+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd000fca0 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.754+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd80c8f10 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.754+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0014660 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.754+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81e68a0 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.755+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd81e4090 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.755+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddd0012040 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.756+0000 7fddd57fa640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 0x7fddb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.756+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fddd008b570 con 0x7fddc40043b0 2024-05-09T08:56:40.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.756+0000 7fddd7fff640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 0x7fddb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:40.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.756+0000 7fddd7fff640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 0x7fddb806e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddc8034000 tx=0x7fddc8009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:40.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.760+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fddd0019030 con 0x7fddc40043b0 2024-05-09T08:56:40.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.938+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddd8068560 con 0x7fddb806be30 2024-05-09T08:56:40.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.945+0000 7fddd57fa640 1 -- 172.21.15.49:0/3465874910 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7fddd8068560 con 0x7fddb806be30 2024-05-09T08:56:40.947 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:40.948 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.948+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 msgr2=0x7fddb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.948+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 0x7fddb806e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddc8034000 tx=0x7fddc8009030 comp rx=0 tx=0).stop 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.948+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 msgr2=0x7fddd815d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.948+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fddd0011aa0 tx=0x7fddd0011f60 comp rx=0 tx=0).stop 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 shutdown_connections 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fddb806be30 0x7fddb806e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd80ca1f0 0x7fddd81e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fddd80c7c70 0x7fddd81e4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 --2- 172.21.15.49:0/3465874910 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fddc40043b0 0x7fddd815d0f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 >> 172.21.15.49:0/3465874910 conn(0x7fddd806bd00 msgr2=0x7fddd815b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 shutdown_connections 2024-05-09T08:56:40.951 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:40.949+0000 7fdddef14640 1 -- 172.21.15.49:0/3465874910 wait complete. 2024-05-09T08:56:40.959 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-05-09T08:56:40.580606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":363,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235784,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738936,"statfs":{"total":773094113280,"available":772852670464,"internally_reserved":0,"allocated":3088384,"data_stored":1764276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651910+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:03.651910+0000","last_peered":"2024-05-09T08:56:03.651910+0000","last_clean":"2024-05-09T08:56:03.651910+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:03.651910+0000","last_undegraded":"2024-05-09T08:56:03.651910+0000","last_fullsized":"2024-05-09T08:56:03.651910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693465+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:56:02.693465+0000","last_peered":"2024-05-09T08:56:02.693465+0000","last_clean":"2024-05-09T08:56:02.693465+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:56:02.693465+0000","last_undegraded":"2024-05-09T08:56:02.693465+0000","last_fullsized":"2024-05-09T08:56:02.693465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043031+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:56:03.043031+0000","last_peered":"2024-05-09T08:56:03.043031+0000","last_clean":"2024-05-09T08:56:03.043031+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:56:03.043031+0000","last_undegraded":"2024-05-09T08:56:03.043031+0000","last_fullsized":"2024-05-09T08:56:03.043031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077190+0000","last_change":"2024-05-09T08:56:01.696722+0000","last_active":"2024-05-09T08:56:03.077190+0000","last_peered":"2024-05-09T08:56:03.077190+0000","last_clean":"2024-05-09T08:56:03.077190+0000","last_became_active":"2024-05-09T08:56:01.695781+0000","last_became_peered":"2024-05-09T08:56:01.695781+0000","last_unstale":"2024-05-09T08:56:03.077190+0000","last_undegraded":"2024-05-09T08:56:03.077190+0000","last_fullsized":"2024-05-09T08:56:03.077190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:11:47.772161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651657+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:03.651657+0000","last_peered":"2024-05-09T08:56:03.651657+0000","last_clean":"2024-05-09T08:56:03.651657+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:03.651657+0000","last_undegraded":"2024-05-09T08:56:03.651657+0000","last_fullsized":"2024-05-09T08:56:03.651657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077235+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:03.077235+0000","last_peered":"2024-05-09T08:56:03.077235+0000","last_clean":"2024-05-09T08:56:03.077235+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:03.077235+0000","last_undegraded":"2024-05-09T08:56:03.077235+0000","last_fullsized":"2024-05-09T08:56:03.077235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651429+0000","last_change":"2024-05-09T08:56:01.697694+0000","last_active":"2024-05-09T08:56:03.651429+0000","last_peered":"2024-05-09T08:56:03.651429+0000","last_clean":"2024-05-09T08:56:03.651429+0000","last_became_active":"2024-05-09T08:56:01.697426+0000","last_became_peered":"2024-05-09T08:56:01.697426+0000","last_unstale":"2024-05-09T08:56:03.651429+0000","last_undegraded":"2024-05-09T08:56:03.651429+0000","last_fullsized":"2024-05-09T08:56:03.651429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:27.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651334+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:03.651334+0000","last_peered":"2024-05-09T08:56:03.651334+0000","last_clean":"2024-05-09T08:56:03.651334+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:03.651334+0000","last_undegraded":"2024-05-09T08:56:03.651334+0000","last_fullsized":"2024-05-09T08:56:03.651334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651940+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:03.651940+0000","last_peered":"2024-05-09T08:56:03.651940+0000","last_clean":"2024-05-09T08:56:03.651940+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:03.651940+0000","last_undegraded":"2024-05-09T08:56:03.651940+0000","last_fullsized":"2024-05-09T08:56:03.651940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005690+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:03.005690+0000","last_peered":"2024-05-09T08:56:03.005690+0000","last_clean":"2024-05-09T08:56:03.005690+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:03.005690+0000","last_undegraded":"2024-05-09T08:56:03.005690+0000","last_fullsized":"2024-05-09T08:56:03.005690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005650+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:03.005650+0000","last_peered":"2024-05-09T08:56:03.005650+0000","last_clean":"2024-05-09T08:56:03.005650+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:03.005650+0000","last_undegraded":"2024-05-09T08:56:03.005650+0000","last_fullsized":"2024-05-09T08:56:03.005650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695698+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:56:02.695698+0000","last_peered":"2024-05-09T08:56:02.695698+0000","last_clean":"2024-05-09T08:56:02.695698+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:56:02.695698+0000","last_undegraded":"2024-05-09T08:56:02.695698+0000","last_fullsized":"2024-05-09T08:56:02.695698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651690+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:03.651690+0000","last_peered":"2024-05-09T08:56:03.651690+0000","last_clean":"2024-05-09T08:56:03.651690+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:03.651690+0000","last_undegraded":"2024-05-09T08:56:03.651690+0000","last_fullsized":"2024-05-09T08:56:03.651690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042880+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:56:03.042880+0000","last_peered":"2024-05-09T08:56:03.042880+0000","last_clean":"2024-05-09T08:56:03.042880+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:56:03.042880+0000","last_undegraded":"2024-05-09T08:56:03.042880+0000","last_fullsized":"2024-05-09T08:56:03.042880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693274+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:56:02.693274+0000","last_peered":"2024-05-09T08:56:02.693274+0000","last_clean":"2024-05-09T08:56:02.693274+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:56:02.693274+0000","last_undegraded":"2024-05-09T08:56:02.693274+0000","last_fullsized":"2024-05-09T08:56:02.693274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077159+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:03.077159+0000","last_peered":"2024-05-09T08:56:03.077159+0000","last_clean":"2024-05-09T08:56:03.077159+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:03.077159+0000","last_undegraded":"2024-05-09T08:56:03.077159+0000","last_fullsized":"2024-05-09T08:56:03.077159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077564+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:03.077564+0000","last_peered":"2024-05-09T08:56:03.077564+0000","last_clean":"2024-05-09T08:56:03.077564+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:03.077564+0000","last_undegraded":"2024-05-09T08:56:03.077564+0000","last_fullsized":"2024-05-09T08:56:03.077564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688394+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:02.688394+0000","last_peered":"2024-05-09T08:56:02.688394+0000","last_clean":"2024-05-09T08:56:02.688394+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:02.688394+0000","last_undegraded":"2024-05-09T08:56:02.688394+0000","last_fullsized":"2024-05-09T08:56:02.688394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651957+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:03.651957+0000","last_peered":"2024-05-09T08:56:03.651957+0000","last_clean":"2024-05-09T08:56:03.651957+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:03.651957+0000","last_undegraded":"2024-05-09T08:56:03.651957+0000","last_fullsized":"2024-05-09T08:56:03.651957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005645+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:03.005645+0000","last_peered":"2024-05-09T08:56:03.005645+0000","last_clean":"2024-05-09T08:56:03.005645+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:03.005645+0000","last_undegraded":"2024-05-09T08:56:03.005645+0000","last_fullsized":"2024-05-09T08:56:03.005645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043123+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:56:03.043123+0000","last_peered":"2024-05-09T08:56:03.043123+0000","last_clean":"2024-05-09T08:56:03.043123+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:56:03.043123+0000","last_undegraded":"2024-05-09T08:56:03.043123+0000","last_fullsized":"2024-05-09T08:56:03.043123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077195+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:03.077195+0000","last_peered":"2024-05-09T08:56:03.077195+0000","last_clean":"2024-05-09T08:56:03.077195+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:03.077195+0000","last_undegraded":"2024-05-09T08:56:03.077195+0000","last_fullsized":"2024-05-09T08:56:03.077195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651411+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:03.651411+0000","last_peered":"2024-05-09T08:56:03.651411+0000","last_clean":"2024-05-09T08:56:03.651411+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:03.651411+0000","last_undegraded":"2024-05-09T08:56:03.651411+0000","last_fullsized":"2024-05-09T08:56:03.651411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077049+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:03.077049+0000","last_peered":"2024-05-09T08:56:03.077049+0000","last_clean":"2024-05-09T08:56:03.077049+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:03.077049+0000","last_undegraded":"2024-05-09T08:56:03.077049+0000","last_fullsized":"2024-05-09T08:56:03.077049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651992+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:03.651992+0000","last_peered":"2024-05-09T08:56:03.651992+0000","last_clean":"2024-05-09T08:56:03.651992+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:03.651992+0000","last_undegraded":"2024-05-09T08:56:03.651992+0000","last_fullsized":"2024-05-09T08:56:03.651992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005730+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:03.005730+0000","last_peered":"2024-05-09T08:56:03.005730+0000","last_clean":"2024-05-09T08:56:03.005730+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:03.005730+0000","last_undegraded":"2024-05-09T08:56:03.005730+0000","last_fullsized":"2024-05-09T08:56:03.005730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693429+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:56:02.693429+0000","last_peered":"2024-05-09T08:56:02.693429+0000","last_clean":"2024-05-09T08:56:02.693429+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:56:02.693429+0000","last_undegraded":"2024-05-09T08:56:02.693429+0000","last_fullsized":"2024-05-09T08:56:02.693429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651904+0000","last_change":"2024-05-09T08:56:01.693942+0000","last_active":"2024-05-09T08:56:03.651904+0000","last_peered":"2024-05-09T08:56:03.651904+0000","last_clean":"2024-05-09T08:56:03.651904+0000","last_became_active":"2024-05-09T08:56:01.693656+0000","last_became_peered":"2024-05-09T08:56:01.693656+0000","last_unstale":"2024-05-09T08:56:03.651904+0000","last_undegraded":"2024-05-09T08:56:03.651904+0000","last_fullsized":"2024-05-09T08:56:03.651904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:26:06.038475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043214+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:56:03.043214+0000","last_peered":"2024-05-09T08:56:03.043214+0000","last_clean":"2024-05-09T08:56:03.043214+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:56:03.043214+0000","last_undegraded":"2024-05-09T08:56:03.043214+0000","last_fullsized":"2024-05-09T08:56:03.043214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077563+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:03.077563+0000","last_peered":"2024-05-09T08:56:03.077563+0000","last_clean":"2024-05-09T08:56:03.077563+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:03.077563+0000","last_undegraded":"2024-05-09T08:56:03.077563+0000","last_fullsized":"2024-05-09T08:56:03.077563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.688313+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:02.688313+0000","last_peered":"2024-05-09T08:56:02.688313+0000","last_clean":"2024-05-09T08:56:02.688313+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:02.688313+0000","last_undegraded":"2024-05-09T08:56:02.688313+0000","last_fullsized":"2024-05-09T08:56:02.688313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005460+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:03.005460+0000","last_peered":"2024-05-09T08:56:03.005460+0000","last_clean":"2024-05-09T08:56:03.005460+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:03.005460+0000","last_undegraded":"2024-05-09T08:56:03.005460+0000","last_fullsized":"2024-05-09T08:56:03.005460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695313+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:56:02.695313+0000","last_peered":"2024-05-09T08:56:02.695313+0000","last_clean":"2024-05-09T08:56:02.695313+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:56:02.695313+0000","last_undegraded":"2024-05-09T08:56:02.695313+0000","last_fullsized":"2024-05-09T08:56:02.695313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.735788+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:11.735788+0000","last_peered":"2024-05-09T08:56:11.735788+0000","last_clean":"2024-05-09T08:56:11.735788+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:11.735788+0000","last_undegraded":"2024-05-09T08:56:11.735788+0000","last_fullsized":"2024-05-09T08:56:11.735788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043238+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:56:03.043238+0000","last_peered":"2024-05-09T08:56:03.043238+0000","last_clean":"2024-05-09T08:56:03.043238+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:56:03.043238+0000","last_undegraded":"2024-05-09T08:56:03.043238+0000","last_fullsized":"2024-05-09T08:56:03.043238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.752061+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:23.752061+0000","last_peered":"2024-05-09T08:56:23.752061+0000","last_clean":"2024-05-09T08:56:23.752061+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:23.752061+0000","last_undegraded":"2024-05-09T08:56:23.752061+0000","last_fullsized":"2024-05-09T08:56:23.752061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.735501+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:17.735501+0000","last_peered":"2024-05-09T08:56:17.735501+0000","last_clean":"2024-05-09T08:56:17.735501+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:17.735501+0000","last_undegraded":"2024-05-09T08:56:17.735501+0000","last_fullsized":"2024-05-09T08:56:17.735501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651388+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:03.651388+0000","last_peered":"2024-05-09T08:56:03.651388+0000","last_clean":"2024-05-09T08:56:03.651388+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:03.651388+0000","last_undegraded":"2024-05-09T08:56:03.651388+0000","last_fullsized":"2024-05-09T08:56:03.651388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.752115+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:23.752115+0000","last_peered":"2024-05-09T08:56:23.752115+0000","last_clean":"2024-05-09T08:56:23.752115+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:23.752115+0000","last_undegraded":"2024-05-09T08:56:23.752115+0000","last_fullsized":"2024-05-09T08:56:23.752115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693269+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:56:02.693269+0000","last_peered":"2024-05-09T08:56:02.693269+0000","last_clean":"2024-05-09T08:56:02.693269+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:56:02.693269+0000","last_undegraded":"2024-05-09T08:56:02.693269+0000","last_fullsized":"2024-05-09T08:56:02.693269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.733031+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:11.733031+0000","last_peered":"2024-05-09T08:56:11.733031+0000","last_clean":"2024-05-09T08:56:11.733031+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:11.733031+0000","last_undegraded":"2024-05-09T08:56:11.733031+0000","last_fullsized":"2024-05-09T08:56:11.733031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077510+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:03.077510+0000","last_peered":"2024-05-09T08:56:03.077510+0000","last_clean":"2024-05-09T08:56:03.077510+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:03.077510+0000","last_undegraded":"2024-05-09T08:56:03.077510+0000","last_fullsized":"2024-05-09T08:56:03.077510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768777+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:29.768777+0000","last_peered":"2024-05-09T08:56:29.768777+0000","last_clean":"2024-05-09T08:56:29.768777+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:29.768777+0000","last_undegraded":"2024-05-09T08:56:29.768777+0000","last_fullsized":"2024-05-09T08:56:29.768777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043098+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:56:03.043098+0000","last_peered":"2024-05-09T08:56:03.043098+0000","last_clean":"2024-05-09T08:56:03.043098+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:56:03.043098+0000","last_undegraded":"2024-05-09T08:56:03.043098+0000","last_fullsized":"2024-05-09T08:56:03.043098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.731426+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:56:11.731426+0000","last_peered":"2024-05-09T08:56:11.731426+0000","last_clean":"2024-05-09T08:56:11.731426+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:56:11.731426+0000","last_undegraded":"2024-05-09T08:56:11.731426+0000","last_fullsized":"2024-05-09T08:56:11.731426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695735+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:56:02.695735+0000","last_peered":"2024-05-09T08:56:02.695735+0000","last_clean":"2024-05-09T08:56:02.695735+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:56:02.695735+0000","last_undegraded":"2024-05-09T08:56:02.695735+0000","last_fullsized":"2024-05-09T08:56:02.695735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'10","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.765633+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:29.765633+0000","last_peered":"2024-05-09T08:56:29.765633+0000","last_clean":"2024-05-09T08:56:29.765633+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:29.765633+0000","last_undegraded":"2024-05-09T08:56:29.765633+0000","last_fullsized":"2024-05-09T08:56:29.765633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651712+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:03.651712+0000","last_peered":"2024-05-09T08:56:03.651712+0000","last_clean":"2024-05-09T08:56:03.651712+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:03.651712+0000","last_undegraded":"2024-05-09T08:56:03.651712+0000","last_fullsized":"2024-05-09T08:56:03.651712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":23,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.766127+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:56:29.766127+0000","last_peered":"2024-05-09T08:56:29.766127+0000","last_clean":"2024-05-09T08:56:29.766127+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:56:29.766127+0000","last_undegraded":"2024-05-09T08:56:29.766127+0000","last_fullsized":"2024-05-09T08:56:29.766127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043106+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:56:03.043106+0000","last_peered":"2024-05-09T08:56:03.043106+0000","last_clean":"2024-05-09T08:56:03.043106+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:56:03.043106+0000","last_undegraded":"2024-05-09T08:56:03.043106+0000","last_fullsized":"2024-05-09T08:56:03.043106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751628+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:23.751628+0000","last_peered":"2024-05-09T08:56:23.751628+0000","last_clean":"2024-05-09T08:56:23.751628+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:23.751628+0000","last_undegraded":"2024-05-09T08:56:23.751628+0000","last_fullsized":"2024-05-09T08:56:23.751628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651736+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:03.651736+0000","last_peered":"2024-05-09T08:56:03.651736+0000","last_clean":"2024-05-09T08:56:03.651736+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:03.651736+0000","last_undegraded":"2024-05-09T08:56:03.651736+0000","last_fullsized":"2024-05-09T08:56:03.651736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751262+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:23.751262+0000","last_peered":"2024-05-09T08:56:23.751262+0000","last_clean":"2024-05-09T08:56:23.751262+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:23.751262+0000","last_undegraded":"2024-05-09T08:56:23.751262+0000","last_fullsized":"2024-05-09T08:56:23.751262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005313+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:03.005313+0000","last_peered":"2024-05-09T08:56:03.005313+0000","last_clean":"2024-05-09T08:56:03.005313+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:03.005313+0000","last_undegraded":"2024-05-09T08:56:03.005313+0000","last_fullsized":"2024-05-09T08:56:03.005313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":31,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.747643+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:23.747643+0000","last_peered":"2024-05-09T08:56:23.747643+0000","last_clean":"2024-05-09T08:56:23.747643+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:23.747643+0000","last_undegraded":"2024-05-09T08:56:23.747643+0000","last_fullsized":"2024-05-09T08:56:23.747643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651464+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:03.651464+0000","last_peered":"2024-05-09T08:56:03.651464+0000","last_clean":"2024-05-09T08:56:03.651464+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:03.651464+0000","last_undegraded":"2024-05-09T08:56:03.651464+0000","last_fullsized":"2024-05-09T08:56:03.651464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.686508+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:56:06.686508+0000","last_peered":"2024-05-09T08:56:06.686508+0000","last_clean":"2024-05-09T08:56:06.686508+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:56:06.686508+0000","last_undegraded":"2024-05-09T08:56:06.686508+0000","last_fullsized":"2024-05-09T08:56:06.686508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651917+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:03.651917+0000","last_peered":"2024-05-09T08:56:03.651917+0000","last_clean":"2024-05-09T08:56:03.651917+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:03.651917+0000","last_undegraded":"2024-05-09T08:56:03.651917+0000","last_fullsized":"2024-05-09T08:56:03.651917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.767451+0000","last_change":"2024-05-09T08:56:00.694150+0000","last_active":"2024-05-09T08:56:29.767451+0000","last_peered":"2024-05-09T08:56:29.767451+0000","last_clean":"2024-05-09T08:56:29.767451+0000","last_became_active":"2024-05-09T08:56:00.693816+0000","last_became_peered":"2024-05-09T08:56:00.693816+0000","last_unstale":"2024-05-09T08:56:29.767451+0000","last_undegraded":"2024-05-09T08:56:29.767451+0000","last_fullsized":"2024-05-09T08:56:29.767451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:48:20.762627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:03.652074+0000","last_peered":"2024-05-09T08:56:03.652074+0000","last_clean":"2024-05-09T08:56:03.652074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:03.652074+0000","last_undegraded":"2024-05-09T08:56:03.652074+0000","last_fullsized":"2024-05-09T08:56:03.652074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750461+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:56:23.750461+0000","last_peered":"2024-05-09T08:56:23.750461+0000","last_clean":"2024-05-09T08:56:23.750461+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:56:23.750461+0000","last_undegraded":"2024-05-09T08:56:23.750461+0000","last_fullsized":"2024-05-09T08:56:23.750461+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077395+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:03.077395+0000","last_peered":"2024-05-09T08:56:03.077395+0000","last_clean":"2024-05-09T08:56:03.077395+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:03.077395+0000","last_undegraded":"2024-05-09T08:56:03.077395+0000","last_fullsized":"2024-05-09T08:56:03.077395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750621+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:23.750621+0000","last_peered":"2024-05-09T08:56:23.750621+0000","last_clean":"2024-05-09T08:56:23.750621+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:23.750621+0000","last_undegraded":"2024-05-09T08:56:23.750621+0000","last_fullsized":"2024-05-09T08:56:23.750621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651700+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:03.651700+0000","last_peered":"2024-05-09T08:56:03.651700+0000","last_clean":"2024-05-09T08:56:03.651700+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:03.651700+0000","last_undegraded":"2024-05-09T08:56:03.651700+0000","last_fullsized":"2024-05-09T08:56:03.651700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651538+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:03.651538+0000","last_peered":"2024-05-09T08:56:03.651538+0000","last_clean":"2024-05-09T08:56:03.651538+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:03.651538+0000","last_undegraded":"2024-05-09T08:56:03.651538+0000","last_fullsized":"2024-05-09T08:56:03.651538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.750728+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:23.750728+0000","last_peered":"2024-05-09T08:56:23.750728+0000","last_clean":"2024-05-09T08:56:23.750728+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:23.750728+0000","last_undegraded":"2024-05-09T08:56:23.750728+0000","last_fullsized":"2024-05-09T08:56:23.750728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651585+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:03.651585+0000","last_peered":"2024-05-09T08:56:03.651585+0000","last_clean":"2024-05-09T08:56:03.651585+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:03.651585+0000","last_undegraded":"2024-05-09T08:56:03.651585+0000","last_fullsized":"2024-05-09T08:56:03.651585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.738287+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:17.738287+0000","last_peered":"2024-05-09T08:56:17.738287+0000","last_clean":"2024-05-09T08:56:17.738287+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:17.738287+0000","last_undegraded":"2024-05-09T08:56:17.738287+0000","last_fullsized":"2024-05-09T08:56:17.738287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042824+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:56:03.042824+0000","last_peered":"2024-05-09T08:56:03.042824+0000","last_clean":"2024-05-09T08:56:03.042824+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:56:03.042824+0000","last_undegraded":"2024-05-09T08:56:03.042824+0000","last_fullsized":"2024-05-09T08:56:03.042824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:17.735438+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:17.735438+0000","last_peered":"2024-05-09T08:56:17.735438+0000","last_clean":"2024-05-09T08:56:17.735438+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:17.735438+0000","last_undegraded":"2024-05-09T08:56:17.735438+0000","last_fullsized":"2024-05-09T08:56:17.735438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693258+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:56:02.693258+0000","last_peered":"2024-05-09T08:56:02.693258+0000","last_clean":"2024-05-09T08:56:02.693258+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:56:02.693258+0000","last_undegraded":"2024-05-09T08:56:02.693258+0000","last_fullsized":"2024-05-09T08:56:02.693258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768265+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:29.768265+0000","last_peered":"2024-05-09T08:56:29.768265+0000","last_clean":"2024-05-09T08:56:29.768265+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:29.768265+0000","last_undegraded":"2024-05-09T08:56:29.768265+0000","last_fullsized":"2024-05-09T08:56:29.768265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695555+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:56:02.695555+0000","last_peered":"2024-05-09T08:56:02.695555+0000","last_clean":"2024-05-09T08:56:02.695555+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:56:02.695555+0000","last_undegraded":"2024-05-09T08:56:02.695555+0000","last_fullsized":"2024-05-09T08:56:02.695555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693296+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:56:02.693296+0000","last_peered":"2024-05-09T08:56:02.693296+0000","last_clean":"2024-05-09T08:56:02.693296+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:56:02.693296+0000","last_undegraded":"2024-05-09T08:56:02.693296+0000","last_fullsized":"2024-05-09T08:56:02.693296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077417+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:03.077417+0000","last_peered":"2024-05-09T08:56:03.077417+0000","last_clean":"2024-05-09T08:56:03.077417+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:03.077417+0000","last_undegraded":"2024-05-09T08:56:03.077417+0000","last_fullsized":"2024-05-09T08:56:03.077417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.749772+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:23.749772+0000","last_peered":"2024-05-09T08:56:23.749772+0000","last_clean":"2024-05-09T08:56:23.749772+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:23.749772+0000","last_undegraded":"2024-05-09T08:56:23.749772+0000","last_fullsized":"2024-05-09T08:56:23.749772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.693393+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:56:02.693393+0000","last_peered":"2024-05-09T08:56:02.693393+0000","last_clean":"2024-05-09T08:56:02.693393+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:56:02.693393+0000","last_undegraded":"2024-05-09T08:56:02.693393+0000","last_fullsized":"2024-05-09T08:56:02.693393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:23.751566+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:56:23.751566+0000","last_peered":"2024-05-09T08:56:23.751566+0000","last_clean":"2024-05-09T08:56:23.751566+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:56:23.751566+0000","last_undegraded":"2024-05-09T08:56:23.751566+0000","last_fullsized":"2024-05-09T08:56:23.751566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.652033+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:03.652033+0000","last_peered":"2024-05-09T08:56:03.652033+0000","last_clean":"2024-05-09T08:56:03.652033+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:03.652033+0000","last_undegraded":"2024-05-09T08:56:03.652033+0000","last_fullsized":"2024-05-09T08:56:03.652033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768513+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:29.768513+0000","last_peered":"2024-05-09T08:56:29.768513+0000","last_clean":"2024-05-09T08:56:29.768513+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:29.768513+0000","last_undegraded":"2024-05-09T08:56:29.768513+0000","last_fullsized":"2024-05-09T08:56:29.768513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.043113+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:56:03.043113+0000","last_peered":"2024-05-09T08:56:03.043113+0000","last_clean":"2024-05-09T08:56:03.043113+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:56:03.043113+0000","last_undegraded":"2024-05-09T08:56:03.043113+0000","last_fullsized":"2024-05-09T08:56:03.043113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.730452+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:11.730452+0000","last_peered":"2024-05-09T08:56:11.730452+0000","last_clean":"2024-05-09T08:56:11.730452+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:11.730452+0000","last_undegraded":"2024-05-09T08:56:11.730452+0000","last_fullsized":"2024-05-09T08:56:11.730452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:02.695419+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:56:02.695419+0000","last_peered":"2024-05-09T08:56:02.695419+0000","last_clean":"2024-05-09T08:56:02.695419+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:56:02.695419+0000","last_undegraded":"2024-05-09T08:56:02.695419+0000","last_fullsized":"2024-05-09T08:56:02.695419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.769188+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:56:29.769188+0000","last_peered":"2024-05-09T08:56:29.769188+0000","last_clean":"2024-05-09T08:56:29.769188+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:56:29.769188+0000","last_undegraded":"2024-05-09T08:56:29.769188+0000","last_fullsized":"2024-05-09T08:56:29.769188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651642+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:03.651642+0000","last_peered":"2024-05-09T08:56:03.651642+0000","last_clean":"2024-05-09T08:56:03.651642+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:03.651642+0000","last_undegraded":"2024-05-09T08:56:03.651642+0000","last_fullsized":"2024-05-09T08:56:03.651642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:24.729364+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:56:24.729364+0000","last_peered":"2024-05-09T08:56:24.729364+0000","last_clean":"2024-05-09T08:56:24.729364+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:56:24.729364+0000","last_undegraded":"2024-05-09T08:56:24.729364+0000","last_fullsized":"2024-05-09T08:56:24.729364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651841+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:03.651841+0000","last_peered":"2024-05-09T08:56:03.651841+0000","last_clean":"2024-05-09T08:56:03.651841+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:03.651841+0000","last_undegraded":"2024-05-09T08:56:03.651841+0000","last_fullsized":"2024-05-09T08:56:03.651841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:11.721112+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:11.721112+0000","last_peered":"2024-05-09T08:56:11.721112+0000","last_clean":"2024-05-09T08:56:11.721112+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:11.721112+0000","last_undegraded":"2024-05-09T08:56:11.721112+0000","last_fullsized":"2024-05-09T08:56:11.721112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.005381+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:03.005381+0000","last_peered":"2024-05-09T08:56:03.005381+0000","last_clean":"2024-05-09T08:56:03.005381+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:03.005381+0000","last_undegraded":"2024-05-09T08:56:03.005381+0000","last_fullsized":"2024-05-09T08:56:03.005381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077292+0000","last_change":"2024-05-09T08:56:01.687323+0000","last_active":"2024-05-09T08:56:03.077292+0000","last_peered":"2024-05-09T08:56:03.077292+0000","last_clean":"2024-05-09T08:56:03.077292+0000","last_became_active":"2024-05-09T08:56:01.686949+0000","last_became_peered":"2024-05-09T08:56:01.686949+0000","last_unstale":"2024-05-09T08:56:03.077292+0000","last_undegraded":"2024-05-09T08:56:03.077292+0000","last_fullsized":"2024-05-09T08:56:03.077292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:48:46.010625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768412+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:29.768412+0000","last_peered":"2024-05-09T08:56:29.768412+0000","last_clean":"2024-05-09T08:56:29.768412+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:29.768412+0000","last_undegraded":"2024-05-09T08:56:29.768412+0000","last_fullsized":"2024-05-09T08:56:29.768412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.077217+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:03.077217+0000","last_peered":"2024-05-09T08:56:03.077217+0000","last_clean":"2024-05-09T08:56:03.077217+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:03.077217+0000","last_undegraded":"2024-05-09T08:56:03.077217+0000","last_fullsized":"2024-05-09T08:56:03.077217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'11","reported_seq":25,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768783+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:29.768783+0000","last_peered":"2024-05-09T08:56:29.768783+0000","last_clean":"2024-05-09T08:56:29.768783+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:29.768783+0000","last_undegraded":"2024-05-09T08:56:29.768783+0000","last_fullsized":"2024-05-09T08:56:29.768783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.042957+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:56:03.042957+0000","last_peered":"2024-05-09T08:56:03.042957+0000","last_clean":"2024-05-09T08:56:03.042957+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:56:03.042957+0000","last_undegraded":"2024-05-09T08:56:03.042957+0000","last_fullsized":"2024-05-09T08:56:03.042957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:06.685096+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:06.685096+0000","last_peered":"2024-05-09T08:56:06.685096+0000","last_clean":"2024-05-09T08:56:06.685096+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:06.685096+0000","last_undegraded":"2024-05-09T08:56:06.685096+0000","last_fullsized":"2024-05-09T08:56:06.685096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:03.651938+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:03.651938+0000","last_peered":"2024-05-09T08:56:03.651938+0000","last_clean":"2024-05-09T08:56:03.651938+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:03.651938+0000","last_undegraded":"2024-05-09T08:56:03.651938+0000","last_fullsized":"2024-05-09T08:56:03.651938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":27,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T08:56:29.768497+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:29.768497+0000","last_peered":"2024-05-09T08:56:29.768497+0000","last_clean":"2024-05-09T08:56:29.768497+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:56:29.768497+0000","last_undegraded":"2024-05-09T08:56:29.768497+0000","last_fullsized":"2024-05-09T08:56:29.768497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":204,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":65,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":663402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":73336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":253952,"data_stored":74866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":72264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":73370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":72396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":72466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":827392,"data_stored":662176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:56:40.966 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T08:56:40.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-09T08:56:41.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.103+0000 7ffa64963640 1 Processor -- start 2024-05-09T08:56:41.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.104+0000 7ffa64963640 1 -- start start 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.104+0000 7ffa64963640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 0x7ffa6015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.104+0000 7ffa64963640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa60075200 0x7ffa600ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.104+0000 7ffa64963640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa600cd520 0x7ffa600cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa64963640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ffa60151bd0 con 0x7ffa600cd520 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa64963640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ffa6015e260 con 0x7ffa601517f0 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa64963640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6015e520 con 0x7ffa60075200 2024-05-09T08:56:41.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa600cd520 0x7ffa600cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ffa600cd520 0x7ffa600cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34440/0 (socket says 172.21.15.49:34440) 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 -- 172.21.15.49:0/4108146508 learned_addr learned my addr 172.21.15.49:0/4108146508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5dd9b640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa60075200 0x7ffa600ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5e59c640 1 --2- 172.21.15.49:0/4108146508 >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 0x7ffa6015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 -- 172.21.15.49:0/4108146508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa60075200 msgr2=0x7ffa600ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa60075200 0x7ffa600ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 -- 172.21.15.49:0/4108146508 >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 msgr2=0x7ffa6015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 --2- 172.21.15.49:0/4108146508 >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 0x7ffa6015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5ed9d640 1 -- 172.21.15.49:0/4108146508 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6015e800 con 0x7ffa600cd520 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.105+0000 7ffa5e59c640 1 --2- 172.21.15.49:0/4108146508 >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 0x7ffa6015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa5ed9d640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600cd520 0x7ffa600cc190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009e50 tx=0x7ffa5802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2198dbba6ea05786 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa5d59a640 1 -- 172.21.15.49:0/4108146508 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58036070 con 0x7ffa600cd520 2024-05-09T08:56:41.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa5d59a640 1 -- 172.21.15.49:0/4108146508 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa58031600 con 0x7ffa600cd520 2024-05-09T08:56:41.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa5d59a640 1 -- 172.21.15.49:0/4108146508 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa580319d0 con 0x7ffa600cd520 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa64963640 1 -- 172.21.15.49:0/4108146508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600cd520 msgr2=0x7ffa600cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.106+0000 7ffa64963640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600cd520 0x7ffa600cc190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009e50 tx=0x7ffa5802a750 comp rx=0 tx=0).stop 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 -- 172.21.15.49:0/4108146508 shutdown_connections 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa600cd520 0x7ffa600cc190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 --2- 172.21.15.49:0/4108146508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa60075200 0x7ffa600ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 --2- 172.21.15.49:0/4108146508 >> v2:172.21.15.183:3300/0 conn(0x7ffa601517f0 0x7ffa6015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 -- 172.21.15.49:0/4108146508 >> 172.21.15.49:0/4108146508 conn(0x7ffa6006bd00 msgr2=0x7ffa60074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 -- 172.21.15.49:0/4108146508 shutdown_connections 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.107+0000 7ffa64963640 1 -- 172.21.15.49:0/4108146508 wait complete. 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.108+0000 7ffa64963640 1 Processor -- start 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.108+0000 7ffa64963640 1 -- start start 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa600cd520 0x7ffa6015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 0x7ffa6015db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6015f0a0 con 0x7ffa60075200 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6015edc0 con 0x7ffa601517f0 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa64963640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffa600cbc70 con 0x7ffa600cd520 2024-05-09T08:56:41.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34456/0 (socket says 172.21.15.49:34456) 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 -- 172.21.15.49:0/1673575580 learned_addr learned my addr 172.21.15.49:0/1673575580 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5dd9b640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa600cd520 0x7ffa6015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5ed9d640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 0x7ffa6015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 -- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa600cd520 msgr2=0x7ffa6015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa600cd520 0x7ffa6015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 -- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 msgr2=0x7ffa6015db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 0x7ffa6015db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 -- 172.21.15.49:0/1673575580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601e5700 con 0x7ffa60075200 2024-05-09T08:56:41.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5ed9d640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 0x7ffa6015db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa5e59c640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c820 tx=0x7ffa5000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.109+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50015070 con 0x7ffa60075200 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.110+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa50007a30 con 0x7ffa60075200 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.110+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa50007de0 con 0x7ffa60075200 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.110+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601f4950 con 0x7ffa60075200 2024-05-09T08:56:41.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.110+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601f0c00 con 0x7ffa60075200 2024-05-09T08:56:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.111+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa601f1040 con 0x7ffa60075200 2024-05-09T08:56:41.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.111+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa50018070 con 0x7ffa60075200 2024-05-09T08:56:41.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.112+0000 7ffa56ffd640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 0x7ffa4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.112+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffa5008ac80 con 0x7ffa60075200 2024-05-09T08:56:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.112+0000 7ffa5dd9b640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 0x7ffa4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.112+0000 7ffa5dd9b640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 0x7ffa4006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa600c7060 tx=0x7ffa48008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.116+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffa500596c0 con 0x7ffa60075200 2024-05-09T08:56:41.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.301+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ffa60068560 con 0x7ffa60075200 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.302+0000 7ffa56ffd640 1 -- 172.21.15.49:0/1673575580 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v20) ==== 56+0+3243 (secure 0 0 0) 0x7ffa50058fc0 con 0x7ffa60075200 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 20 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stdout:fsid aab8b2de-6010-439c-b8fd-8bc72445265c 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-05-09T08:55:44.445697+0000 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-05-09T08:56:02.682459+0000 2024-05-09T08:56:41.301 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 4 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client jewel 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release squid 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.50 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T08:56:41.302 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 8 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] [v2:172.21.15.49:6802/1154213557,v1:172.21.15.49:6803/1154213557] exists,up b59d26bf-6806-4e49-b117-9d83d3a24243 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] [v2:172.21.15.49:6818/3346547076,v1:172.21.15.49:6819/3346547076] exists,up d579c6a2-0370-4c21-96fb-6420583dbe74 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] [v2:172.21.15.49:6810/136930210,v1:172.21.15.49:6811/136930210] exists,up bb2ad0e5-2ec1-4647-bc42-3d20911e038a 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] [v2:172.21.15.49:6826/1536339711,v1:172.21.15.49:6827/1536339711] exists,up 546a2935-7144-4c29-b647-e77d89072d60 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] [v2:172.21.15.183:6810/173400960,v1:172.21.15.183:6811/173400960] exists,up f4bb828b-6036-4886-aec7-be53c00ac603 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] [v2:172.21.15.183:6802/4279483611,v1:172.21.15.183:6803/4279483611] exists,up 41625c33-9f16-4279-b2aa-a63ac0aa6320 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] [v2:172.21.15.183:6826/3521264583,v1:172.21.15.183:6827/3521264583] exists,up 9177552d-bc69-409e-9654-e030c3a1afba 2024-05-09T08:56:41.303 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] [v2:172.21.15.183:6818/2743896476,v1:172.21.15.183:6819/2743896476] exists,up d3b1821c-1ed4-4286-b7de-030fda235fbf 2024-05-09T08:56:41.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 msgr2=0x7ffa4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 0x7ffa4006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa600c7060 tx=0x7ffa48008040 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 msgr2=0x7ffa601e51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c820 tx=0x7ffa5000cce0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 shutdown_connections 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa4006be30 0x7ffa4006e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa601517f0 0x7ffa6015db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa600cd520 0x7ffa6015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 --2- 172.21.15.49:0/1673575580 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa60075200 0x7ffa601e51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 >> 172.21.15.49:0/1673575580 conn(0x7ffa6006bd00 msgr2=0x7ffa600c5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 shutdown_connections 2024-05-09T08:56:41.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.304+0000 7ffa64963640 1 -- 172.21.15.49:0/1673575580 wait complete. 2024-05-09T08:56:41.314 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] [v2:172.21.15.49:6802/1154213557,v1:172.21.15.49:6803/1154213557] exists,up b59d26bf-6806-4e49-b117-9d83d3a24243', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] [v2:172.21.15.49:6818/3346547076,v1:172.21.15.49:6819/3346547076] exists,up d579c6a2-0370-4c21-96fb-6420583dbe74', 'osd.2 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] [v2:172.21.15.49:6810/136930210,v1:172.21.15.49:6811/136930210] exists,up bb2ad0e5-2ec1-4647-bc42-3d20911e038a', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] [v2:172.21.15.49:6826/1536339711,v1:172.21.15.49:6827/1536339711] exists,up 546a2935-7144-4c29-b647-e77d89072d60', 'osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] [v2:172.21.15.183:6810/173400960,v1:172.21.15.183:6811/173400960] exists,up f4bb828b-6036-4886-aec7-be53c00ac603', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] [v2:172.21.15.183:6802/4279483611,v1:172.21.15.183:6803/4279483611] exists,up 41625c33-9f16-4279-b2aa-a63ac0aa6320', 'osd.6 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] [v2:172.21.15.183:6826/3521264583,v1:172.21.15.183:6827/3521264583] exists,up 9177552d-bc69-409e-9654-e030c3a1afba', 'osd.7 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] [v2:172.21.15.183:6818/2743896476,v1:172.21.15.183:6819/2743896476] exists,up d3b1821c-1ed4-4286-b7de-030fda235fbf'] 2024-05-09T08:56:41.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mon.a", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster": "ceph", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "admin_socket_mode": "", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_client_required": "cephx, none", 2024-05-09T08:56:41.420 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_cluster_required": "cephx", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_debug": "true", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_service_required": "cephx", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_supported": "cephx", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio": "true", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_poll_ms": "250", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_reap_max": "16", 2024-05-09T08:56:41.421 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_async_discard": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_block_size": "4096", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_inflight_ios": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_enable_discard": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_flock_retry": "3", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_inject_crash": "0", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring_hipri": "false", 2024-05-09T08:56:41.422 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_type": "", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_alloc_size": "1048576", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_allocator": "hybrid", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_buffered_io": "true", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_for_zeros": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_compact_log_sync": "false", 2024-05-09T08:56:41.423 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_max_log_runway": "4194304", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_max_prefetch": "1048576", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_min_flush_size": "524288", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_min_log_runway": "1048576", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_replay_recovery": "false", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_sync_write": "false", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-09T08:56:41.424 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_allocation_from_file": "true", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_allocator": "hybrid", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-09T08:56:41.425 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_create": "true", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_create": "false", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_path": "", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_size": "0", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_path": "", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_preallocate_file": "false", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_size": "107374182400", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_create": "false", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_path": "", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_size": "100663296", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs": "true", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-09T08:56:41.426 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_autotune": "true", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size": "0", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-09T08:56:41.427 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_type": "2q", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_clone_cow": "true", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-09T08:56:41.428 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_mode": "none", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_csum_type": "crc32c", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-09T08:56:41.429 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_freelist": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-09T08:56:41.430 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_small_allocations": "0", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_default_buffered_read": "true", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_default_buffered_write": "false", 2024-05-09T08:56:41.431 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fail_eio": "false", 2024-05-09T08:56:41.432 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mount": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_umount": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-09T08:56:41.433 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_ignore_data_csum": "false", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-09T08:56:41.434 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_op_age": "5.000000", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_alloc_size": "0", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size": "0", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_deferred_txc": "32", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size": "0", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_nid_prealloc": "1024", 2024-05-09T08:56:41.435 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_retry_disk_reads": "3", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_cf": "true", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-09T08:56:41.436 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_mem": "512", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_tracing": "false", 2024-05-09T08:56:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_zero_block_detection": "false", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-09T08:56:41.438 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-09T08:56:41.439 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_cluster_require_signatures": "false", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_cluster_require_version": "2", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_require_signatures": "false", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_require_version": "2", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_service_require_signatures": "false", 2024-05-09T08:56:41.440 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_service_require_version": "2", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_sign_messages": "true", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "chdir": "", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_acl_type": "", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_asio_thread_count": "2", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_cache_mid": "0.750000", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_cache_size": "16384", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_caps_release_delay": "5", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_check_pool_perm": "true", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_force_sync_read": "false", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_getattr_caps": "false", 2024-05-09T08:56:41.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_inject_tick_delay": "0", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_die_on_failed_remount": "false", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_dirsize_rbytes": "true", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_force_lazyio": "false", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_fs": "", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_inject_release_failure": "false", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_max_inline_size": "4096", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mds_namespace": "", 2024-05-09T08:56:41.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_metadata": "", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_gid": "-1", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_timeout": "300", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_uid": "-1", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mountpoint": "/", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_notify_timeout": "10", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc": "true", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_dirty": "104857600", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_objects": "1000", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_size": "209715200", 2024-05-09T08:56:41.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_target_dirty": "8388608", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_permissions": "true", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_quota": "true", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_quota_df": "true", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_max_bytes": "0", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_max_periods": "4", 2024-05-09T08:56:41.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_min": "131072", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_reconnect_stale": "false", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_shutdown_timeout": "30", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_snapdir": ".snap", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_tick_interval": "1", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_trace": "", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_try_dentry_invalidate": "false", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_use_faked_inos": "false", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_use_random_mds": "false", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog": "false", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog_port": "12201", 2024-05-09T08:56:41.445 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_monitors": "default=true", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog": "false", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog_level": "info", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "-", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_network": "", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_network_interface": "", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_isal": "false", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_level": "5", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_winsize": "-15", 2024-05-09T08:56:41.446 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zstd_level": "1", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_alien_op_num_threads": "6", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_seastar_cpu_cores": "", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location": "", 2024-05-09T08:56:41.447 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location_hook": "", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location_hook_timeout": "10", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "daemonize": "false", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "daos_pool": "tank", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_alienstore": "0/5", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_allow_any_pool_priority": "false", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asok": "1/5", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asok_assert_abort": "false", 2024-05-09T08:56:41.448 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asserts_on_shutdown": "true", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_auth": "20/20", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bdev": "1/3", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bluefs": "1/5", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bluestore": "1/5", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_buffer": "0/1", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_ceph_exporter": "1/5", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_cephfs_mirror": "0/5", 2024-05-09T08:56:41.449 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_cephsqlite": "0/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_client": "0/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_compressor": "1/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_context": "0/1", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_crimson_interrupt": "0/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_crush": "1/1", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_crypto": "1/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_cyanstore": "0/5", 2024-05-09T08:56:41.450 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_deliberately_leak_memory": "false", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_disable_randomized_ping": "false", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_dpdk": "1/5", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_eventtrace": "1/5", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_filer": "0/1", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_filestore": "1/3", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_finisher": "1/1", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_fuse": "1/5", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_heartbeat_testing_span": "0", 2024-05-09T08:56:41.451 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_heartbeatmap": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_javaclient": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_journal": "1/3", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_journaler": "0/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_kstore": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_lockdep": "0/1", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mclock": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_balancer": "1/5", 2024-05-09T08:56:41.452 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_locker": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_log": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_log_expire": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_migrator": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_quiesce": "3/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_memstore": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mgr": "2/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mgrc": "1/5", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mon": "20/20", 2024-05-09T08:56:41.453 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_monc": "0/10", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_ms": "1/1", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_none": "0/5", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objclass": "0/5", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objectcacher": "0/5", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objecter": "0/1", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_optracker": "0/5", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_osd": "1/5", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_paxos": "20/20", 2024-05-09T08:56:41.454 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_perfcounter": "1/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_prioritycache": "1/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rados": "0/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd": "0/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_mirror": "0/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_pwl": "0/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_replay": "0/5", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_refs": "0/0", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_reserver": "1/1", 2024-05-09T08:56:41.455 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_access": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_datacache": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_dbstore": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_flight": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_sync": "1/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rocksdb": "4/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore": "0/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_backref": "0/5", 2024-05-09T08:56:41.456 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_cache": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_cleaner": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_device": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_epm": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_journal": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_lba": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_odata": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_omap": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_onode": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_t": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_tm": "0/5", 2024-05-09T08:56:41.457 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_striper": "0/1", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_test": "0/5", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_throttle": "1/1", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_timer": "0/1", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_tp": "0/5", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_trace": "1/5", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "device_failure_prediction_mode": "none", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-09T08:56:41.458 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_graylog": "false", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_journald": "false", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_stderr": "true", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_syslog": "false", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event_tracing": "false", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_addr": "0.0.0.0", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_http_port": "9926", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_prio_limit": "5", 2024-05-09T08:56:41.459 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_sort_metrics": "true", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_stats_period": "5", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "fake_statfs_for_testing": "0", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "fatal_signal_handlers": "true", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filer_max_purge_ops": "10", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filer_max_truncate_ops": "128", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_apply_finisher_threads": "1", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_blackhole": "false", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_btrfs_clone_range": "true", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_btrfs_snap": "true", 2024-05-09T08:56:41.460 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_caller_concurrency": "10", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_collect_device_partition_information": "true", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_commit_timeout": "600.000000", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_inject_read_err": "false", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_omap_check": "false", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_verify_split": "false", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_dump_file": "", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-09T08:56:41.461 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fadvise": "true", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fail_eio": "true", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fd_cache_shards": "16", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fd_cache_size": "128", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fiemap": "false", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fiemap_threshold": "4096", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_inject_stall": "0", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_parallel": "false", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_trailing": "false", 2024-05-09T08:56:41.462 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_writeahead": "false", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_kill_at": "0", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs": "0", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-09T08:56:41.463 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size": "0", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_merge_threshold": "-10", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_odsync_write": "false", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_backend": "rocksdb", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_backend_path": "", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-09T08:56:41.464 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_thread_timeout": "60", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_threads": "2", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_punch_hole": "false", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-09T08:56:41.465 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_ops": "50", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_seek_data_hole": "false", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_sloppy_crc": "false", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_splice": "false", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_split_multiple": "2", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_split_rand_factor": "20", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_update_to": "1000", 2024-05-09T08:56:41.466 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_enable": "true", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-09T08:56:41.467 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_xfs_extsize": "false", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_zfs_snap": "false", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fio_dir": "/tmp/fio", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_allow_other": "true", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_atomic_o_trunc": "true", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_big_writes": "true", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_debug": "false", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_default_permissions": "false", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_disable_pagecache": "false", 2024-05-09T08:56:41.468 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_max_write": "0", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_multithreaded": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_require_active_mds": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_set_user_groups": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_move": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_read": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_write": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_use_invalidate_cb": "true", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "gss_target_name": "ceph", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_file": "", 2024-05-09T08:56:41.469 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_inject_failure": "0", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_interval": "5", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "host": "smithi049", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-09T08:56:41.470 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "inject_early_sigterm": "false", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "jaeger_agent_port": "6799", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "jaeger_tracing_enable": "false", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_aio": "true", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_align_min_size": "65536", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_block_align": "true", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_block_size": "4096", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_dio": "true", 2024-05-09T08:56:41.471 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_discard": "false", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_force_aio": "false", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_ignore_corruption": "false", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_max_write_bytes": "10485760", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_max_write_entries": "100", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_replay_from": "0", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_write_header_frequency": "0", 2024-05-09T08:56:41.472 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_zero_on_create": "false", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_prefetch_periods": "10", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_prezero_periods": "5", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_write_head_interval": "15", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "keyfile": "", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_backend": "rocksdb", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_default_stripe_size": "65536", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_fsck_on_mount": "false", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-09T08:56:41.473 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_max_bytes": "67108864", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_max_ops": "512", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_nid_prealloc": "1024", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_onode_map_size": "1024", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_sync_submit_transaction": "false", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_sync_transaction": "false", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "librados_thread_count": "2", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "lockdep": "false", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "lockdep_force_backtrace": "false", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "log_coarse_timestamps": "true", 2024-05-09T08:56:41.474 INFO:teuthology.orchestra.run.smithi049.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_flush_on_exit": "false", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_graylog_host": "127.0.0.1", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_graylog_port": "12201", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_max_new": "1000", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_max_recent": "10000", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_stderr_prefix": "", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_stop_at_utilization": "0.970000", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": "true", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_graylog": "false", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_journald": "false", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_stderr": "false", 2024-05-09T08:56:41.475 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_syslog": "false", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "max_rotating_auth_attempts": "10", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_action_on_write_error": "1", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_alternate_name_max": "8192", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_asio_thread_count": "2", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_export_pin": "true", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_dirs": "true", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_interval": "5", 2024-05-09T08:56:41.476 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_interval": "10", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_max": "-1", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_max_until": "-1", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_merge_size": "50", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_midchunk": "0.300000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_min_start": "0.200000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_minchunk": "0.001000", 2024-05-09T08:56:41.477 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_mode": "0", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_need_max": "1.200000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_need_min": "0.800000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_overload_epochs": "2", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_bits": "3", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_size": "10000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-09T08:56:41.478 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_target_decay": "10.000000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_grace": "15.000000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_interval": "4.000000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_mid": "0.700000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-09T08:56:41.479 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_quiesce_delay": "0", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_release_free_interval": "10", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_reservation": "0.050000", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_interval": "1", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_threshold": "262144", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-09T08:56:41.480 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_prealloc_inos": "1000", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_connect_bootstrapping": "false", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_damage_table_max_entries": "10000", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_auth_pins": "false", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_frag": "false", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_scatterstat": "false", 2024-05-09T08:56:41.481 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_subtrees": "false", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_decay_halflife": "5.000000", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_default_dir_hash": "2", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_defer_session_stale": "true", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_deny_all_reconnect": "false", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_keys_per_op": "16384", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_max_commit_size": "10", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_max_entries": "0", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_prefetch": "true", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-09T08:56:41.482 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_on_map": "false", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_early_reply": "true", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_enable_op_tracker": "true", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_enforce_unique_name": "true", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-09T08:56:41.483 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_random": "true", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-09T08:56:41.484 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_health_summarize_threshold": "10", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_health_dummy": "false", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_migrator_session_race": "false", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-09T08:56:41.485 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_join_fs": "", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_journal_format": "1", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_create_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_export_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_import_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_expire_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_replay_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_link_at": "0", 2024-05-09T08:56:41.486 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_mdstable_at": "0", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_openc_at": "0", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_rename_at": "0", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_shutdown_at": "0", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_event_large_threshold": "524288", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_events_per_segment": "1024", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_max_events": "-1", 2024-05-09T08:56:41.487 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_max_segments": "128", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_pause": "false", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_segment_size": "0", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_trim_threshold": "128", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_warn_factor": "2.000000", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_caps_per_client": "1048576", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_completed_flushes": "100000", 2024-05-09T08:56:41.488 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_completed_requests": "100000", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_export_size": "20971520", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_file_recover": "32", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_files": "64", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_ops": "8192", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_snaps_per_dir": "100", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_metrics_update_interval": "2", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_min_caps_per_client": "100", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_min_caps_working_set": "10000", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-09T08:56:41.489 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_numa_node": "-1", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_complaint_time": "30.000000", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_duration": "600", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_size": "20", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_slow_op_size": "20", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_log_threshold": "5", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_ping_grace": "15", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_ping_interval": "5", 2024-05-09T08:56:41.490 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_caps": "30000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_warning_threshold": "262144", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_replay_interval": "1.000000", 2024-05-09T08:56:41.491 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_root_ino_gid": "0", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_root_ino_uid": "0", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-09T08:56:41.492 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_shutdown_check": "0", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_skip_ino": "0", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_max_uid": "4294967294", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_min_uid": "0", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_rstat": "false", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_standby_replay_damaged": "false", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_symlink_recovery": "true", 2024-05-09T08:56:41.493 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_thrash_exports": "0", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_thrash_fragments": "0", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_tick_interval": "5.000000", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_valgrind_exit": "false", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_verify_backtrace": "1", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_verify_scatter": "false", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_wipe_sessions": "false", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool_debug": "false", 2024-05-09T08:56:41.494 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_device_bytes": "1073741824", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_page_set": "false", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_page_size": "65536", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_bytes": "134217728", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_messages": "512", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_disabled_modules": "", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-09T08:56:41.495 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_max_pg_num_change": "128", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mds_bytes": "134217728", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mds_messages": "128", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mon_bytes": "134217728", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mon_messages": "128", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_osd_bytes": "536870912", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_osd_messages": "8192", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_pool": "true", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-09T08:56:41.496 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_standby_modules": "true", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_stats_period": "5", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_stats_threshold": "5", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_tick_period": "2", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_allow_pool_delete": "true", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_allow_pool_size_one": "true", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_auth_validate_all_caps": "true", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_bytes": "104857600", 2024-05-09T08:56:41.497 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_directed_command_retry": "2", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_parallel": "3", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_log_interval": "1.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_ping_interval": "10.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_target_rank": "-1", 2024-05-09T08:56:41.498 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_file_level": "debug", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_file": "true", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_journald": "false", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-09T08:56:41.499 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_bootstrap": "false", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_start": "false", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_trim": "true", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cpu_threads": "4", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_crush_min_required_version": "hammer", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_daemon_bytes": "419430400", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-09T08:56:41.500 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_avail_crit": "5", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_avail_warn": "5", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_size_warn": "16106127360", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_json": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_transactions": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_extra_checks": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-09T08:56:41.501 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_reef": "false", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_squid": "false", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_down_mkfs_grace": "300", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_down_uptime_grace": "60", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_election_default_strategy": "1", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_election_timeout": "5.000000", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ping_divisor": "2", 2024-05-09T08:56:41.502 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_enable_op_tracker": "true", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_fake_pool_delete": "false", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_force_quorum_join": "false", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_globalid_prealloc": "10000", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_detail_to_clog": "false", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_log_update_period": "5", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_max_detail": "50", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog": "true", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog_interval": "600", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_host": "172.21.15.49,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]", 2024-05-09T08:56:41.503 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_host_override": "", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_initial_members": "", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease": "5.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-09T08:56:41.504 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_full_interval": "50", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_max": "10000", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_max_summary": "50", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_log_epochs": "500", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_osd": "10000", 2024-05-09T08:56:41.505 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_pg_per_osd": "10000", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_pool_pg_num": "65536", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_force_trim_to": "0", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_skip_sanity": "false", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_memory_autotune": "true", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_memory_target": "2147483648", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_beacon_grace": "30", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_digest_period": "5", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_inactive_grace": "60", 2024-05-09T08:56:41.506 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_min_osdmap_epochs": "500", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_complaint_time": "120", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_duration": "600", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_size": "20", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_slow_op_size": "20", 2024-05-09T08:56:41.507 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_log_threshold": "5", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_in": "false", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_cache_size": "500", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-09T08:56:41.508 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_down_out_interval": "600", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_force_trim_to": "0", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-09T08:56:41.509 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_down_reporters": "2", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-09T08:56:41.510 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_report_timeout": "900", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-09T08:56:41.511 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_interval": "10", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_min": "10000", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_stuck_threshold": "60", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-09T08:56:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_probe_timeout": "2.000000", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_max_change": "0.050000", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_max_osds": "4", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-09T08:56:41.513 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_interval": "86400", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_max_keys": "100", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_timeout": "300", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_session_timeout": "300", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_smart_report_timeout": "5", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stat_smooth_intervals": "6", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-09T08:56:41.514 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_pool_min_size": "2", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_pool_size": "4", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_debug": "false", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_provider_kill_at": "0", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_requester_kill_at": "0", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_timeout": "60.000000", 2024-05-09T08:56:41.515 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_target_pg_per_osd": "100", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_tick_interval": "5", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_timecheck_interval": "300.000000", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_use_min_delay_socket": "false", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_older_version_delay": "604800", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-09T08:56:41.516 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_misplaced": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_older_version": "true", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_no_app": "true", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-09T08:56:41.517 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "monmap": "", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-09T08:56:41.518 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "motr_tracing_enabled": "false", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_op_threads": "3", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_cm": "false", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_device_name": "", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_dscp": "96", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-09T08:56:41.519 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_local_gid": "", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_port_num": "1", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_sl": "3", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_support_srq": "true", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_type": "ib", 2024-05-09T08:56:41.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_reap_threshold": "5", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_before_connect": "false", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_ipv4": "true", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_ipv6": "false", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_msgr1": "true", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_msgr2": "true", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_port_max": "7568", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_port_min": "6800", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_retry_count": "3", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_retry_delay": "5", 2024-05-09T08:56:41.521 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_client": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mds": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mgr": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mon": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_osd": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_client_mode": "crc secure", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_cluster_mode": "crc secure", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_cluster_type": "", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_compress_secure": "false", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_connection_idle_timeout": "900", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_connection_ready_timeout": "10", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_crc_data": "true", 2024-05-09T08:56:41.522 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_crc_header": "true", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_bad_msg": "false", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_bug": "true", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_old_message": "true", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_skipped_message": "false", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_coremask": "0xF", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_enable_tso": "true", 2024-05-09T08:56:41.523 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hugepages": "", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_lro": "true", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_memory_channel": "4", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_pmd": "", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_port_id": "0", 2024-05-09T08:56:41.524 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dump_on_send": "false", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_initial_backoff": "0.200000", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_max": "1.000000", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_type": "", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_network_congestion": "0", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_socket_failures": "0", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_learn_addr_from_peer": "true", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_max_accept_failures": "4", 2024-05-09T08:56:41.525 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_max_backoff": "15.000000", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_client_mode": "secure crc", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_service_mode": "secure crc", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compress_min_size": "1024", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compress_mode": "none", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_pq_min_cost": "65536", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_public_type": "", 2024-05-09T08:56:41.526 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_service_mode": "crc secure", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_listen_backlog": "512", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_nodelay": "true", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_rcvbuf": "0", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_type": "async+posix", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "no_config_file": "false", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_completion_locks_per_session": "32", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inflight_ops": "1024", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-09T08:56:41.527 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_tick_interval": "5.000000", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_timeout": "10.000000", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "objectstore_blackhole": "false", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "openssl_engine_opts": "", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_delay_time": "5.000000", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_hist_halflife": "1000", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_max_low_ops": "2", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_max_ops": "4", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-09T08:56:41.528 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_slop": "0.020000", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_asio_thread_count": "2", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_async_recovery_min_cost": "100", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_scan_max": "512", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_scan_min": "64", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_degraded": "false", 2024-05-09T08:56:41.529 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_peering": "false", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_unfound": "true", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_beacon_report_interval": "300", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_duration": "30", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_max_block_size": "67108864", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_blkin_trace_all": "false", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-09T08:56:41.530 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_check_for_log_corruption": "false", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_update_on_start": "true", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_message_cap": "256", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_message_size_cap": "524288000", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_op_priority": "63", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_watch_timeout": "30", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_command_max_records": "256", 2024-05-09T08:56:41.531 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_compact_on_start": "false", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_update_on_start": "true", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_update_weight_set": "true", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_feed_pullee": "-1", 2024-05-09T08:56:41.532 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_misdirected_ops": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_no_acting_change": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_no_purge_strays": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_op_order": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-09T08:56:41.533 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_shutdown": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_trim_objects": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-09T08:56:41.534 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_snaps": "false", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_keys": "1024", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_stride": "524288", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_default_notify_timeout": "30", 2024-05-09T08:56:41.535 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep": "0.000000", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_discard_disconnected_ops": "true", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_enable_op_tracker": "true", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-09T08:56:41.536 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_info": "true", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown": "true", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_function_tracing": "false", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_grace": "20", 2024-05-09T08:56:41.537 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_interval": "6", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_peers": "10", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_size": "2000", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_stale": "600", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_max_size": "100000", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_min_size": "1000", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-09T08:56:41.538 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal_size": "5120", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_kill_backfill_at": "0", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_cache_size": "50", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_dedup": "true", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_message_max": "40", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_share_max_epochs": "40", 2024-05-09T08:56:41.539 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_attr_name_len": "100", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_attr_size": "0", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_backfills": "1", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_markdown_count": "5", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_markdown_period": "600", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_name_len": "2048", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_namespace_len": "256", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_size": "134217728", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_blocked_by": "16", 2024-05-09T08:56:41.540 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_log_entries": "10000", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pgls": "1024", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_push_cost": "8388608", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_push_objects": "10", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_scrubs": "3", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_trimming_pgs": "2", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_write_op_reply_len": "64", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_write_size": "90", 2024-05-09T08:56:41.541 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_profile": "balanced", 2024-05-09T08:56:41.542 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-09T08:56:41.543 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_base": "805306368", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_cache_min": "134217728", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target": "4294967296", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target_autotune": "false", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_min_pg_log_entries": "250", 2024-05-09T08:56:41.544 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_min_recovery_priority": "0", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_report_interval": "5", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_num_cache_shards": "32", 2024-05-09T08:56:41.545 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_num_op_tracker_shard": "32", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_auto_affinity": "true", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_node": "-1", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_prefer_iface": "true", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objecter_finishers": "1", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore": "bluestore", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore_fuse": "false", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore_tracing": "false", 2024-05-09T08:56:41.546 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_complaint_time": "30.000000", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_duration": "600", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_size": "20", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_slow_op_size": "20", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_log_threshold": "5", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards": "0", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards_hdd": "5", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards_ssd": "8", 2024-05-09T08:56:41.547 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_pq_min_cost": "65536", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_queue_cut_off": "high", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-09T08:56:41.548 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_thread_timeout": "15", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_open_classes_on_start": "true", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_os_flags": "0", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_peering_op_priority": "255", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_delete_cost": "1048576", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_delete_priority": "5", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_trim_max": "10000", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_trim_min": "100", 2024-05-09T08:56:41.549 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_crimson": "false", 2024-05-09T08:56:41.550 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flags": "0", 2024-05-09T08:56:41.551 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_min_size": "0", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pg_num": "32", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pgp_num": "0", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_read_ratio": "70", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_size": "2", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_type": "replicated", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-09T08:56:41.552 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_push_per_object_cost": "1000", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recover_clone_overlap": "true", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_cost": "20971520", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active": "0", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-09T08:56:41.553 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_single_start": "1", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_op_priority": "3", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_priority": "5", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep": "0.000000", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_repair_during_recovery": "false", 2024-05-09T08:56:41.554 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_requested_scrub_priority": "120", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_auto_repair": "false", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_begin_hour": "0", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_begin_week_day": "0", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_chunk_max": "25", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_chunk_min": "5", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_cost": "52428800", 2024-05-09T08:56:41.555 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_during_recovery": "false", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_end_hour": "0", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_end_week_day": "0", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_event_cost": "4096", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_invalid_stats": "true", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_max_preemptions": "5", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-09T08:56:41.556 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_priority": "5", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_sleep": "0.000000", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_skip_data_digest": "false", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_smart_report_timeout": "5", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_cost": "1048576", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_priority": "5", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-09T08:56:41.557 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_target_transaction_size": "30", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-09T08:56:41.558 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tracing": "false", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_use_stale_snap": "false", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "osdc_blkin_trace_all": "false", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_kill_at": "0", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_max_join_drift": "10", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_min": "500", 2024-05-09T08:56:41.559 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_min_wait": "0.050000", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_propose_interval": "1.000000", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_max": "500", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_min": "250", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_stash_full_interval": "25", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_trim_max": "500", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_trim_min": "250", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "perf": "true", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-09T08:56:41.560 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "-", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrv": "", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "public_bind_addr": "-", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "public_network": "", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "public_network_interface": "", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "qat_compressor_enabled": "false", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "qat_compressor_session_max_number": "256", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_mon_op_timeout": "0", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_osd_op_timeout": "0", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_tracing": "false", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_atime_update_interval": "60", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-09T08:56:41.561 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_balance_parent_reads": "false", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_balance_snap_reads": "false", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blkin_trace_all": "false", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache": "true", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_policy": "writearound", 2024-05-09T08:56:41.562 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_size": "33554432", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_clone_copy_on_read": "false", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_compression_hint": "none", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_concurrent_management_ops": "10", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_clone_format": "auto", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_data_pool": "", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_features": "61", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_format": "2", 2024-05-09T08:56:41.563 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_map_options": "", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_order": "22", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_pool": "rbd", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_stripe_count": "0", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_stripe_unit": "0", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_enable_alloc_hint": "true", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-09T08:56:41.564 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_io_scheduler": "simple", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-09T08:56:41.565 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_order": "24", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_pool": "", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_splay_width": "4", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_localize_parent_reads": "false", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_localize_snap_reads": "false", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-09T08:56:41.566 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-09T08:56:41.567 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-09T08:56:41.568 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mtime_update_interval": "60", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_non_blocking_aio": "true", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_op_thread_timeout": "60", 2024-05-09T08:56:41.569 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_op_threads": "1", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_parent_cache_enabled": "false", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_plugins": "", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_burst": "0", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-09T08:56:41.570 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_limit": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_exclude_ops": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_burst": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_limit": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-09T08:56:41.571 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-09T08:56:41.572 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_read_from_replica_policy": "default", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_skip_partial_discard": "true", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_tracing": "false", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_validate_names": "true", 2024-05-09T08:56:41.573 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_validate_pool": "true", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "restapi_base_url": "", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "restapi_log_level": "", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_acl_grants_max_num": "100", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_admin_entry": "admin", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_backend_store": "rados", 2024-05-09T08:56:41.574 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_barbican_url": "", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_beast_enable_async": "true", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_counters_cache": "false", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-09T08:56:41.575 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_enabled": "true", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_expiry_interval": "900", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_lru_size": "25000", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_config_store": "rados", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_content_length_compat": "false", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_copy_obj_progress": "true", 2024-05-09T08:56:41.576 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cors_rules_max_num": "100", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cross_domain_policy": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_addr": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_password": "", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-09T08:56:41.577 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_username": "", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_require_ssl": "true", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-09T08:56:41.578 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_addr": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_auth": "token", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_namespace": "", 2024-05-09T08:56:41.579 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_prefix": "", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_token_file": "", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_buffersize": "524288", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_low_speed_time": "300", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-09T08:56:41.580 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-09T08:56:41.581 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d4n_port": "6379", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_changes_size": "1000", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_num_shards": "128", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_window": "30", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-09T08:56:41.582 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-09T08:56:41.583 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dns_name": "", 2024-05-09T08:56:41.584 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dns_s3website_name": "", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dynamic_resharding": "true", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_gc_threads": "true", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_lc_threads": "true", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_ops_log": "false", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_quota_threads": "true", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_static_website": "false", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_usage_log": "false", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enforce_swift_acls": "true", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_exit_timeout_secs": "120", 2024-05-09T08:56:41.585 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_expose_bucket": "false", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_extended_http_attrs": "", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_filter": "none", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_frontends": "beast port=7480", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_deferred": "50", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_objs": "32", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-09T08:56:41.586 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_processor_period": "3600", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_init_timeout": "300", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_kafka_connection_idle": "300", 2024-05-09T08:56:41.587 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_domain": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_password": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_password_path": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_project": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_tenant": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_token": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_token_path": "", 2024-05-09T08:56:41.588 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_user": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_api_version": "2", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_domain": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_password": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_project": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_user": "", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-09T08:56:41.589 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_url": "", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_debug_interval": "-1", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_lock_max_time": "90", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_objs": "32", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_rules": "1000", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_worker": "3", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_thread_delay": "0", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-09T08:56:41.590 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_dnattr": "uid", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_searchfilter": "", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_uri": "", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_http_headers": "", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-09T08:56:41.591 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_object_name_utc": "false", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attr_name_len": "0", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attr_size": "0", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_chunk_size": "4194304", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_listing_results": "1000", 2024-05-09T08:56:41.592 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_notify_retries": "10", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_put_param_size": "1048576", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_put_size": "5368709120", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_slo_entries": "1000", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_md_log_max_shards": "64", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_mp_lock_max_time": "600", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-09T08:56:41.593 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_max_gc": "300", 2024-05-09T08:56:41.594 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_num_async_rados_threads": "32", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_num_control_oids": "8", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_numa_node": "-1", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-09T08:56:41.595 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_chunk_size": "100", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_gc_interval": "600", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_thread_timeout": "600", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_tracing": "false", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_token": "", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_url": "", 2024-05-09T08:56:41.596 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_verify_ssl": "true", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_rados": "false", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_socket_path": "", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_push_interval": "2.000000", 2024-05-09T08:56:41.597 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_cache_lanes": "3", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_cache_partitions": "3", 2024-05-09T08:56:41.598 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_print_continue": "true", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_tracing": "false", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm": "", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm_id": "", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_region": "", 2024-05-09T08:56:41.599 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_request_uri": "", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_batch_size": "64", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_max_aio": "128", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_num_logs": "16", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_thread_interval": "600", 2024-05-09T08:56:41.600 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_resolve_cname": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_run_sync_thread": "true", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-09T08:56:41.601 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_scheduler_type": "throttler", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_script_uri": "", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_service_provider_name": "", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_client_id": "", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_client_secret": "", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_entry": "sts", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_key": "sts", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_min_session_duration": "900", 2024-05-09T08:56:41.602 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_token_introspection_url": "", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_account_in_url": "false", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_auth_entry": "auth", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_auth_url": "", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_custom_header": "", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_need_stats": "true", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_tenant_name": "", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_token_expiration": "86400", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_url": "", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_url_prefix": "swift", 2024-05-09T08:56:41.603 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_lease_period": "120", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-09T08:56:41.604 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_thread_pool_size": "512", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-09T08:56:41.605 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_comment": "", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_createby": "", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_encoding": "", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_flag": "false", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_origin": "", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_tracker": "", 2024-05-09T08:56:41.606 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_trust_forwarded_https": "false", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_max_shards": "32", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_max_user_shards": "1", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_use_opa_authz": "false", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_counters_cache": "false", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-09T08:56:41.607 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_max_buckets": "1000", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_policies_max_num": "100", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_unique_email": "true", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_verify_ssl": "true", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone": "", 2024-05-09T08:56:41.608 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone_id": "", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup": "", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup_id": "", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgwlc_auto_session_clear": "true", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_block_size": "4096", 2024-05-09T08:56:41.609 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_size": "536870912", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-09T08:56:41.610 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_index_type": "binary_search", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_partition_filters": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_perf": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-09T08:56:41.611 INFO:teuthology.orchestra.run.smithi049.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "run_dir": "/var/run/ceph", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_block_create": "true", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_cache_lru_size": "67108864", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_default_max_object_size": "16777216", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-09T08:56:41.612 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_device_size": "53687091200", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_capacity": "16", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_main_device_type": "SSD", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-09T08:56:41.613 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_segment_size": "67108864", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "set_keepcaps": "false", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "setgroup": "", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "setuser": "", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "setuser_match_path": "", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "thp": "false", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "threadpool_default_timeout": "60", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout: "throttler_perf_counter": "true" 2024-05-09T08:56:41.614 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:41.617 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-09T08:56:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.721+0000 7f75c6789640 1 Processor -- start 2024-05-09T08:56:41.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.721+0000 7f75c6789640 1 -- start start 2024-05-09T08:56:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f75c0154e40 0x7f75c006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75c00c9430 0x7f75c00ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f75c015bc70 con 0x7f75c00c9430 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f75c015b6b0 con 0x7f75c0154e40 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c6789640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75c00c8dc0 con 0x7f75c00c9b60 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c4d26640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75c00c9430 0x7f75c00ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c4d26640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75c00c9430 0x7f75c00ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34464/0 (socket says 172.21.15.49:34464) 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.722+0000 7f75c4d26640 1 -- 172.21.15.49:0/1359137644 learned_addr learned my addr 172.21.15.49:0/1359137644 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 -- 172.21.15.49:0/1359137644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 msgr2=0x7f75c006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 -- 172.21.15.49:0/1359137644 >> v2:172.21.15.183:3300/0 conn(0x7f75c0154e40 msgr2=0x7f75c006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 --2- 172.21.15.49:0/1359137644 >> v2:172.21.15.183:3300/0 conn(0x7f75c0154e40 0x7f75c006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 -- 172.21.15.49:0/1359137644 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c00c9060 con 0x7f75c00c9430 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.723+0000 7f75c4d26640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c00c9430 0x7f75c00ca0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f75c00cf440 tx=0x7f75b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e22e2c6f0486ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75beffd640 1 -- 172.21.15.49:0/1359137644 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8036070 con 0x7f75c00c9430 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75beffd640 1 -- 172.21.15.49:0/1359137644 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b8031600 con 0x7f75c00c9430 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75beffd640 1 -- 172.21.15.49:0/1359137644 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b80319d0 con 0x7f75c00c9430 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 -- 172.21.15.49:0/1359137644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c00c9430 msgr2=0x7f75c00ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c00c9430 0x7f75c00ca0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f75c00cf440 tx=0x7f75b802a610 comp rx=0 tx=0).stop 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 -- 172.21.15.49:0/1359137644 shutdown_connections 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c00c9430 0x7f75c00ca0b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 --2- 172.21.15.49:0/1359137644 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 --2- 172.21.15.49:0/1359137644 >> v2:172.21.15.183:3300/0 conn(0x7f75c0154e40 0x7f75c006f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.728 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 -- 172.21.15.49:0/1359137644 >> 172.21.15.49:0/1359137644 conn(0x7f75c00cae20 msgr2=0x7f75c00cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 -- 172.21.15.49:0/1359137644 shutdown_connections 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.724+0000 7f75c6789640 1 -- 172.21.15.49:0/1359137644 wait complete. 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.725+0000 7f75c6789640 1 Processor -- start 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 -- start start 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c00c9430 0x7f75c0151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c01e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 0x7f75c01f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75c0070b80 con 0x7f75c0154e40 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75c00c98c0 con 0x7f75c00c9430 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c6789640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75c0155220 con 0x7f75c00c9b60 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c4d26640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 0x7f75c01f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c4d26640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 0x7f75c01f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34468/0 (socket says 172.21.15.49:34468) 2024-05-09T08:56:41.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.726+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 learned_addr learned my addr 172.21.15.49:0/2843262115 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c01e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75bffff640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c00c9430 0x7f75c0151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 msgr2=0x7f75c01e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00c9b60 0x7f75c01e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c00c9430 msgr2=0x7f75c0151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c00c9430 0x7f75c0151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c01f4830 con 0x7f75c0154e40 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c4d26640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 0x7f75c01f3f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75b802ac00 tx=0x7f75b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8041020 con 0x7f75c0154e40 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b8007800 con 0x7f75c0154e40 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8036040 con 0x7f75c0154e40 2024-05-09T08:56:41.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c0150810 con 0x7f75c0154e40 2024-05-09T08:56:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.727+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c01e57c0 con 0x7f75c0154e40 2024-05-09T08:56:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.728+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f75c0069060 con 0x7f75c0154e40 2024-05-09T08:56:41.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.729+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f75b8044020 con 0x7f75c0154e40 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.729+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f75ac06be30 0x7f75ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.729+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f75b80abb10 con 0x7f75c0154e40 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.729+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f75c0069060 con 0x7f75c0154e40 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.729+0000 7f75bffff640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f75ac06be30 0x7f75ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 0x7f75c0154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75c004f660 con 0x7f75c0153cd0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 0x7f75c0154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f75c006d030 con 0x7f75c0153cd0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bffff640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f75ac06be30 0x7f75ac06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b810 tx=0x7f75b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 0x7f75c0154160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f75c015f2c0 tx=0x7f75b400f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4018030 con 0x7f75c0153cd0 2024-05-09T08:56:41.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75b4004020 con 0x7f75c0153cd0 2024-05-09T08:56:41.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 msgr2=0x7f75c0154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 0x7f75c0154160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f75c015f2c0 tx=0x7f75b400f8a0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 msgr2=0x7f75c0154160 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.730+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0153cd0 0x7f75c0154160 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f75c015f2c0 tx=0x7f75b400f8a0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.737+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 0x7f75c014ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.737+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75c00c84f0 con 0x7f75c014aaa0 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.737+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 0x7f75c014ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.737+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f75c00008d0 con 0x7f75c014aaa0 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.737+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 0x7f75c014ae80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75b400bfd0 tx=0x7f75b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.738+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b402c0a0 con 0x7f75c014aaa0 2024-05-09T08:56:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.0 v2:172.21.15.49:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f75b4014a80 con 0x7f75c014aaa0 2024-05-09T08:56:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 msgr2=0x7f75c014ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 0x7f75c014ae80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75b400bfd0 tx=0x7f75b40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 msgr2=0x7f75c014ae80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c014aaa0 0x7f75c014ae80 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75b400bfd0 tx=0x7f75b40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.739 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a: {} 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 0x7f75c014f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.739+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01f4440 con 0x7f75c014b3c0 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.740+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 0x7f75c014f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.740+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f75c0153860 con 0x7f75c014b3c0 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.740+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 0x7f75c014f660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b4008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.740+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b403a070 con 0x7f75c014b3c0 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.1 v2:172.21.15.183:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75b4031040 con 0x7f75c014b3c0 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 msgr2=0x7f75c014f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 0x7f75c014f660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b4008040 comp rx=0 tx=0).stop 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 msgr2=0x7f75c014f660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 reap_dead start 2024-05-09T08:56:41.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.741+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c014b3c0 0x7f75c014f660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.747+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 0x7f75c01e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.747+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01f45f0 con 0x7f75c0151560 2024-05-09T08:56:41.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.747+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 0x7f75c01e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.748+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f75c01509e0 con 0x7f75c0151560 2024-05-09T08:56:41.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.748+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 0x7f75c01e5280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75b401ef00 tx=0x7f75b402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.748+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4040030 con 0x7f75c0151560 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.1 v2:172.21.15.183:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f75b40063d0 con 0x7f75c0151560 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 msgr2=0x7f75c01e5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 0x7f75c01e5280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75b401ef00 tx=0x7f75b402a000 comp rx=0 tx=0).stop 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 msgr2=0x7f75c01e5280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75c0151560 0x7f75c01e5280 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75b401ef00 tx=0x7f75b402a000 comp rx=0 tx=0).stop 2024-05-09T08:56:41.749 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b: {} 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 0x7f75c014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75c0150bf0 con 0x7f75c014fba0 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 0x7f75c014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.750+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f75c01504c0 con 0x7f75c014fba0 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 0x7f75c014ff80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75b400bc30 tx=0x7f75b4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4040050 con 0x7f75c014fba0 2024-05-09T08:56:41.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.2 v2:172.21.15.183:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75b403c830 con 0x7f75c014fba0 2024-05-09T08:56:41.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 msgr2=0x7f75c014ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 0x7f75c014ff80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f75b400bc30 tx=0x7f75b4008040 comp rx=0 tx=0).stop 2024-05-09T08:56:41.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 msgr2=0x7f75c014ff80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.751+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c014fba0 0x7f75c014ff80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.757+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 0x7f75c00cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.757+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75c00ccdb0 con 0x7f75c00cc3a0 2024-05-09T08:56:41.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.758+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 0x7f75c00cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.758+0000 7f75bf7fe640 1 -- 172.21.15.49:0/2843262115 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f75c00ccfb0 con 0x7f75c00cc3a0 2024-05-09T08:56:41.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.758+0000 7f75bf7fe640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 0x7f75c00cc780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f75b4019040 tx=0x7f75b4044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.759+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b403a040 con 0x7f75c00cc3a0 2024-05-09T08:56:41.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 <== mon.2 v2:172.21.15.183:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f75b4057760 con 0x7f75c00cc3a0 2024-05-09T08:56:41.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 msgr2=0x7f75c00cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 0x7f75c00cc780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f75b4019040 tx=0x7f75b4044000 comp rx=0 tx=0).stop 2024-05-09T08:56:41.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75bcff9640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 msgr2=0x7f75c00cc780 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:56:41.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75bcff9640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75c00cc3a0 0x7f75c00cc780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.760 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c: {} 2024-05-09T08:56:41.761 INFO:teuthology.orchestra.run.smithi049.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:56:41.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f75ac06be30 msgr2=0x7f75ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f75ac06be30 0x7f75ac06e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b810 tx=0x7f75b0008040 comp rx=0 tx=0).stop 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 msgr2=0x7f75c01f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 --2- 172.21.15.49:0/2843262115 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75c0154e40 0x7f75c01f3f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75b802ac00 tx=0x7f75b8004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c4d26640 1 -- 172.21.15.49:0/2843262115 reap_dead start 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 shutdown_connections 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 >> 172.21.15.49:0/2843262115 conn(0x7f75c00cae20 msgr2=0x7f75c0159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 shutdown_connections 2024-05-09T08:56:41.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.760+0000 7f75c6789640 1 -- 172.21.15.49:0/2843262115 wait complete. 2024-05-09T08:56:41.771 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-05-09T08:56:41.788 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-09T08:56:41.789 DEBUG:teuthology.orchestra.run.smithi049:> type ceph-objectstore-tool 2024-05-09T08:56:41.804 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-09T08:56:41.804 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T08:56:41.817 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:56:41.817 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T08:56:41.817 INFO:tasks.workunit:timeout=3h 2024-05-09T08:56:41.817 INFO:tasks.workunit:cleanup=True 2024-05-09T08:56:41.818 DEBUG:teuthology.orchestra.run.smithi049:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:41.818 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-09T08:56:41.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:56:41.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:56:41.819 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-09T08:56:41.819 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-09T08:56:41.819 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-09T08:56:41.820 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-09T08:56:41.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:56:41.820 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-09T08:56:41.820 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:56:41.821 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-09T08:56:41.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:56:41.830 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:41.831+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-05-09 08:56:06.677180620 +0000 2024-05-09T08:56:41.864 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-05-09 08:56:38.678298380 +0000 2024-05-09T08:56:41.865 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: 2024-05-09 08:56:06.677180620 +0000 2024-05-09T08:56:41.865 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T08:56:41.865 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T08:56:41.948 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:41.949+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.959+0000 7f49e6215640 1 Processor -- start 2024-05-09T08:56:41.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.959+0000 7f49e6215640 1 -- start start 2024-05-09T08:56:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49e6215640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 0x7f49e006b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49e6215640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49e00c8d20 0x7f49e015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49e6215640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e0072d90 con 0x7f49e015b2a0 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49e6215640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f49e0072ac0 con 0x7f49e00c8d20 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49e6215640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f49e0070700 con 0x7f49e0154c70 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49dffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f49e015b2a0 0x7f49e015b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35254/0 (socket says 172.21.15.49:35254) 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49dffff640 1 -- 172.21.15.49:0/2157563820 learned_addr learned my addr 172.21.15.49:0/2157563820 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:41.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7eff22370640 1 Processor -- start 2024-05-09T08:56:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7eff22370640 1 -- start start 2024-05-09T08:56:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.960+0000 7f49deffd640 1 --2- 172.21.15.49:0/2157563820 >> v2:172.21.15.49:3300/0 conn(0x7f49e00c8d20 0x7f49e015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49df7fe640 1 --2- 172.21.15.49:0/2157563820 >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 0x7f49e006b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f5a7a9c1640 1 Processor -- start 2024-05-09T08:56:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f5a7a9c1640 1 -- start start 2024-05-09T08:56:41.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49deffd640 1 -- 172.21.15.49:0/2157563820 >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 msgr2=0x7f49e006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49deffd640 1 --2- 172.21.15.49:0/2157563820 >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 0x7f49e006b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49de7fc640 1 -- 172.21.15.49:0/2157563820 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2930364834 0 0) 0x7f49e0072d90 con 0x7f49e015b2a0 2024-05-09T08:56:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49deffd640 1 -- 172.21.15.49:0/2157563820 >> v1:172.21.15.49:6789/0 conn(0x7f49e015b2a0 legacy=0x7f49e015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49deffd640 1 -- 172.21.15.49:0/2157563820 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e00709e0 con 0x7f49e00c8d20 2024-05-09T08:56:41.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.961+0000 7f49df7fe640 1 --2- 172.21.15.49:0/2157563820 >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 0x7f49e006b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff22370640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7eff1c0c7a50 0x7eff1c067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff22370640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c0c91f0 con 0x7eff1c15b410 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff22370640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff1c0c8f10 con 0x7eff1c15b030 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7eff1c15b410 0x7eff1c0c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35256/0 (socket says 172.21.15.49:35256) 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff22370640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7eff1c06ef20 con 0x7eff1c0c7a50 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1467072992 learned_addr learned my addr 172.21.15.49:0/1467072992 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:41.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a7a9c1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5a7415b410 0x7f5a74067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff2090d640 1 --2- 172.21.15.49:0/1467072992 >> v2:172.21.15.183:3300/0 conn(0x7eff1c0c7a50 0x7eff1c067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242982481 0 0) 0x7eff1c0c91f0 con 0x7eff1c15b410 2024-05-09T08:56:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff04003520 con 0x7eff1c15b410 2024-05-09T08:56:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 225063435 0 0) 0x7eff1c0c8f10 con 0x7eff1c15b030 2024-05-09T08:56:41.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff04003880 con 0x7eff1c15b030 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a7a9c1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a740c91f0 con 0x7f5a74073360 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a7a9c1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a740c8f10 con 0x7f5a7415b030 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a7a9c1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5a7406ef20 con 0x7f5a7415b410 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a73fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f5a7415b030 0x7f5a74154be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:53188/0 (socket says 172.21.15.49:53188) 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a73fff640 1 -- 172.21.15.49:0/2803931645 learned_addr learned my addr 172.21.15.49:0/2803931645 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:41.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f5a737fe640 1 --2- 172.21.15.49:0/2803931645 >> v2:172.21.15.49:3300/0 conn(0x7f5a7415b410 0x7f5a74067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.962+0000 7f49deffd640 1 --2- 172.21.15.49:0/2157563820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e00c8d20 0x7f49e015ad60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f49e0073280 tx=0x7f49d402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b588263f2775ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995071878 0 0) 0x7eff04003880 con 0x7eff1c15b030 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 >> v2:172.21.15.183:3300/0 conn(0x7eff1c0c7a50 msgr2=0x7eff1c067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 --2- 172.21.15.49:0/1467072992 >> v2:172.21.15.183:3300/0 conn(0x7eff1c0c7a50 0x7eff1c067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 >> v1:172.21.15.49:6789/0 conn(0x7eff1c15b410 legacy=0x7eff1c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c15b8f0 con 0x7eff1c15b030 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7eff0c001670 con 0x7eff1c15b030 2024-05-09T08:56:41.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 >> v1:172.21.15.183:6789/0 conn(0x7eff1c15b030 legacy=0x7eff1c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff04003d60 0x7eff04004140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274373733 0 0) 0x7f5a740c8f10 con 0x7f5a7415b030 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a5c003520 con 0x7f5a7415b030 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff040052d0 0x7eff0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c06ef20 con 0x7eff04004800 2024-05-09T08:56:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff0400c1d0 con 0x7eff040052d0 2024-05-09T08:56:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff0400c3d0 con 0x7eff04003d60 2024-05-09T08:56:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 920626928 0 0) 0x7f5a5c003520 con 0x7f5a7415b030 2024-05-09T08:56:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f49de7fc640 1 -- 172.21.15.49:0/2157563820 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4037070 con 0x7f49e00c8d20 2024-05-09T08:56:41.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f49de7fc640 1 -- 172.21.15.49:0/2157563820 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d4032440 con 0x7f49e00c8d20 2024-05-09T08:56:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 >> v2:172.21.15.49:3300/0 conn(0x7f5a7415b410 msgr2=0x7f5a74067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 --2- 172.21.15.49:0/2803931645 >> v2:172.21.15.49:3300/0 conn(0x7f5a7415b410 0x7f5a74067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 >> v1:172.21.15.49:6789/0 conn(0x7f5a74073360 legacy=0x7f5a740c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7415b8f0 con 0x7f5a7415b030 2024-05-09T08:56:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f5a64001670 con 0x7f5a7415b030 2024-05-09T08:56:41.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff2090d640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff040052d0 0x7eff0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a737fe640 1 --2- 172.21.15.49:0/2803931645 >> v2:172.21.15.49:3300/0 conn(0x7f5a7415b410 0x7f5a74067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:41.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7eff1bfff640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff04003d60 0x7eff04004140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 >> v1:172.21.15.183:6789/0 conn(0x7f5a7415b030 legacy=0x7f5a74154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a5c0043b0 0x7f5a5c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 0x7f5a5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7406ef20 con 0x7f5a5c005080 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c00bf30 con 0x7f5a5c0039b0 2024-05-09T08:56:41.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5a5c00c130 con 0x7f5a5c0043b0 2024-05-09T08:56:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f49de7fc640 1 -- 172.21.15.49:0/2157563820 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4032810 con 0x7f49e00c8d20 2024-05-09T08:56:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.963+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 0x7f5a5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a737fe640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a5c0043b0 0x7f5a5c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff04003d60 msgr2=0x7eff04004140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff04003d60 0x7eff04004140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff040052d0 msgr2=0x7eff0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff040052d0 0x7eff0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 -- 172.21.15.49:0/1467072992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c15b8f0 con 0x7eff04004800 2024-05-09T08:56:41.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff2090d640 1 -- 172.21.15.49:0/1467072992 reap_dead start 2024-05-09T08:56:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a73fff640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 -- 172.21.15.49:0/2157563820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e00c8d20 msgr2=0x7f49e015ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 --2- 172.21.15.49:0/2157563820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e00c8d20 0x7f49e015ad60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f49e0073280 tx=0x7f49d402a250 comp rx=0 tx=0).stop 2024-05-09T08:56:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a78f5e640 1 -- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a5c0043b0 msgr2=0x7f5a5c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 -- 172.21.15.49:0/2157563820 shutdown_connections 2024-05-09T08:56:41.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a5c0043b0 0x7f5a5c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a78f5e640 1 -- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c0039b0 msgr2=0x7f5a5c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a78f5e640 1 -- 172.21.15.49:0/2803931645 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7415b8f0 con 0x7f5a5c005080 2024-05-09T08:56:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f5a73fff640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c0039b0 0x7f5a5c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:41.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 --2- 172.21.15.49:0/2157563820 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e00c8d20 0x7f49e015ad60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 --2- 172.21.15.49:0/2157563820 >> v2:172.21.15.183:3300/0 conn(0x7f49e0154c70 0x7f49e006b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 -- 172.21.15.49:0/2157563820 >> 172.21.15.49:0/2157563820 conn(0x7f49e0074840 msgr2=0x7f49e0074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 -- 172.21.15.49:0/2157563820 shutdown_connections 2024-05-09T08:56:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7f49e6215640 1 -- 172.21.15.49:0/2157563820 wait complete. 2024-05-09T08:56:41.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.964+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff04004c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff10007fd0 tx=0x7eff04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3385b66dd101bdf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7f5a78f5e640 1 -- 172.21.15.49:0/2803931645 reap_dead start 2024-05-09T08:56:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10007ba0 con 0x7eff04004800 2024-05-09T08:56:41.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 0x7f5a5c00b9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001f30 tx=0x7f5a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8738826201102c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff10005b90 con 0x7eff04004800 2024-05-09T08:56:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7eff1affd640 1 -- 172.21.15.49:0/1467072992 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff10007ba0 con 0x7eff04004800 2024-05-09T08:56:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c001b30 con 0x7f5a5c005080 2024-05-09T08:56:41.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6c003b90 con 0x7f5a5c005080 2024-05-09T08:56:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.965+0000 7f5a72ffd640 1 -- 172.21.15.49:0/2803931645 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c005110 con 0x7f5a5c005080 2024-05-09T08:56:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 msgr2=0x7f5a5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 0x7f5a5c00b9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001f30 tx=0x7f5a5c005830 comp rx=0 tx=0).stop 2024-05-09T08:56:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f49e6215640 1 Processor -- start 2024-05-09T08:56:41.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 -- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 msgr2=0x7eff04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/2803931645 shutdown_connections 2024-05-09T08:56:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/2803931645 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a5c005080 0x7f5a5c00b9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/2803931645 >> 172.21.15.49:0/2803931645 conn(0x7f5a740c9440 msgr2=0x7f5a740c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff04004c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff10007fd0 tx=0x7eff04005830 comp rx=0 tx=0).stop 2024-05-09T08:56:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/2803931645 shutdown_connections 2024-05-09T08:56:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 -- 172.21.15.49:0/1467072992 shutdown_connections 2024-05-09T08:56:41.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/2803931645 wait complete. 2024-05-09T08:56:41.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7f49e6215640 1 -- start start 2024-05-09T08:56:41.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 --2- 172.21.15.49:0/1467072992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff04004c80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 -- 172.21.15.49:0/1467072992 >> 172.21.15.49:0/1467072992 conn(0x7eff1c0c9440 msgr2=0x7eff1c0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:41.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 -- 172.21.15.49:0/1467072992 shutdown_connections 2024-05-09T08:56:41.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.966+0000 7eff22370640 1 -- 172.21.15.49:0/1467072992 wait complete. 2024-05-09T08:56:41.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.967+0000 7f49e6215640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 0x7f49e01f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.967+0000 7f5a7a9c1640 1 Processor -- start 2024-05-09T08:56:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.967+0000 7f49e6215640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 0x7f49e01559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.967+0000 7f5a7a9c1640 1 -- start start 2024-05-09T08:56:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49df7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 0x7f49e01f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49df7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 0x7f49e01f0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:38572/0 (socket says 172.21.15.49:38572) 2024-05-09T08:56:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49df7fe640 1 -- 172.21.15.49:0/2904836391 learned_addr learned my addr 172.21.15.49:0/2904836391 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49deffd640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 0x7f49e01559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c005080 0x7f5a740cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 0x7f49e014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 0x7f5a740c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7414d8d0 0x7f5a7414dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74067a80 con 0x7f5a7414d8d0 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7406ebb0 con 0x7f5a5c005080 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a7a9c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5a74067c00 con 0x7f5a741f5d80 2024-05-09T08:56:41.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 0x7f5a740c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a73fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c005080 0x7f5a740cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a737fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 0x7f5a740c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:38576/0 (socket says 172.21.15.49:38576) 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a737fe640 1 -- 172.21.15.49:0/3648563334 learned_addr learned my addr 172.21.15.49:0/3648563334 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49dffff640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 0x7f49e014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49e006d140 con 0x7f49e015b2a0 2024-05-09T08:56:41.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49e006fff0 con 0x7f49e0154c70 2024-05-09T08:56:41.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.968+0000 7f5a78f5e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7414d8d0 0x7f5a7414dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f49e015c250 con 0x7f49e00c8d20 2024-05-09T08:56:41.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7eff22370640 1 Processor -- start 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c005080 msgr2=0x7f5a740cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a5c005080 0x7f5a740cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7414d8d0 msgr2=0x7f5a7414dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5a7414d8d0 0x7f5a7414dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a740cae40 con 0x7f5a741f5d80 2024-05-09T08:56:41.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7eff22370640 1 -- start start 2024-05-09T08:56:41.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 0x7f5a740c3f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011af0 tx=0x7f5a6800f240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800fc20 con 0x7f5a741f5d80 2024-05-09T08:56:41.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6800fdc0 con 0x7f5a741f5d80 2024-05-09T08:56:41.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49deffd640 1 -- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 msgr2=0x7f49e01f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49deffd640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 0x7f49e01f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49deffd640 1 -- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 msgr2=0x7f49e014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:41.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a740cb0e0 con 0x7f5a741f5d80 2024-05-09T08:56:41.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004260 con 0x7f5a741f5d80 2024-05-09T08:56:41.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49deffd640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 0x7f49e014f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:41.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f49deffd640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e0158f30 con 0x7f49e0154c70 2024-05-09T08:56:41.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7f49dffff640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 0x7f49e014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:41.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.969+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a740cc240 con 0x7f5a741f5d80 2024-05-09T08:56:41.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7f49deffd640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 0x7f49e01559b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49e0073600 tx=0x7f49d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:41.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4041030 con 0x7f49e0154c70 2024-05-09T08:56:41.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff22370640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff22370640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c1582c0 0x7eff1c1f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d4030770 con 0x7f49e0154c70 2024-05-09T08:56:41.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff22370640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c1415b0 0x7eff1c070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:41.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:41.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff1bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:34536/0 (socket says 172.21.15.49:34536) 2024-05-09T08:56:41.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e0159210 con 0x7f49e0154c70 2024-05-09T08:56:41.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.970+0000 7eff1bfff640 1 -- 172.21.15.49:0/7180130 learned_addr learned my addr 172.21.15.49:0/7180130 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:41.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c073360 con 0x7eff04004800 2024-05-09T08:56:42.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c1582c0 0x7eff1c1f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c06ebb0 con 0x7eff1c1415b0 2024-05-09T08:56:42.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff1c0734e0 con 0x7eff1c1582c0 2024-05-09T08:56:42.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5a7404f660 con 0x7f5a741f5d80 2024-05-09T08:56:42.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff2090d640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c1415b0 0x7eff1c070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4037040 con 0x7f49e0154c70 2024-05-09T08:56:42.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e01597d0 con 0x7f49e0154c70 2024-05-09T08:56:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 -- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c1582c0 msgr2=0x7eff1c1f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c1582c0 0x7eff1c1f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 -- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c1415b0 msgr2=0x7eff1c070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c1415b0 0x7eff1c070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c1556c0 con 0x7eff04004800 2024-05-09T08:56:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff1bfff640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e7c0 tx=0x7eff0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.971+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0052e0 con 0x7eff04004800 2024-05-09T08:56:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff0c018070 con 0x7eff04004800 2024-05-09T08:56:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a68012040 con 0x7f5a741f5d80 2024-05-09T08:56:42.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e006c4c0 con 0x7f49e0154c70 2024-05-09T08:56:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00c900 con 0x7eff04004800 2024-05-09T08:56:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c1559a0 con 0x7eff04004800 2024-05-09T08:56:42.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1f43b0 con 0x7eff04004800 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7f5a70ff9640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5a6006be60 0x7f5a6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.972+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a6808b220 con 0x7f5a741f5d80 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v20) ==== 54+0+16 (secure 0 0 0) 0x7f5a68019030 con 0x7f5a741f5d80 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5a6006be60 0x7f5a6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5a741589b0 0x7f5a740c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a74154670 con 0x7f5a741589b0 2024-05-09T08:56:42.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49d4044070 con 0x7f49e0154c70 2024-05-09T08:56:42.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5a741589b0 0x7f5a740c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.973+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5a6006be60 0x7f5a6006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a64001eb0 tx=0x7f5a64007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f49c7fff640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 0x7f49cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5a741589b0 0x7f5a740c4cd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f49d40ab610 con 0x7f49e0154c70 2024-05-09T08:56:42.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f49df7fe640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 0x7f49cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c04f660 con 0x7eff04004800 2024-05-09T08:56:42.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff0c00caa0 con 0x7eff04004800 2024-05-09T08:56:42.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a74154670 con 0x7f5a741589b0 2024-05-09T08:56:42.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.974+0000 7f49df7fe640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 0x7f49cc06e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009560 tx=0x7f49d0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.975+0000 7eff18ff9640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.975+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7eff0c08ac20 con 0x7eff04004800 2024-05-09T08:56:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.975+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.976+0000 7eff1b7fe640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 0x7eff0806e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff1002f000 tx=0x7eff1002d000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.978+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49e006c4c0 con 0x7f49e0154c70 2024-05-09T08:56:42.011 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:42.011 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:42.011 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:42.011 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:42.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.980+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff0c058ee0 con 0x7eff04004800 2024-05-09T08:56:42.013 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:56:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:41.998+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a740c91f0 con 0x7f5a741589b0 2024-05-09T08:56:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.000+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a740c91f0 con 0x7f5a741589b0 2024-05-09T08:56:42.013 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.000+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5a74158d90 0x7f5a741f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.000+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5a74067860 con 0x7f5a74158d90 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.000+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5a74158d90 0x7f5a741f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.000+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5a74158d90 0x7f5a741f8870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.001+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a74067860 con 0x7f5a74158d90 2024-05-09T08:56:42.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.016+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a74159b90 con 0x7f5a74158d90 2024-05-09T08:56:42.018 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:42.018 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.018+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a74159b90 con 0x7f5a74158d90 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.018+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f5a741f92c0 0x7f5a74061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.018+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5a741598b0 con 0x7f5a741f92c0 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.018+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f5a741f92c0 0x7f5a74061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.019+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f5a741f92c0 0x7f5a74061c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.020+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a741598b0 con 0x7f5a741f92c0 2024-05-09T08:56:42.024 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:42.025+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:42.032 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T08:56:42.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.038+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a74062400 con 0x7f5a741f92c0 2024-05-09T08:56:42.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.041+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a74062400 con 0x7f5a741f92c0 2024-05-09T08:56:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.041+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5a740629d0 0x7f5a740c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.041+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5a740c7420 con 0x7f5a740629d0 2024-05-09T08:56:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.041+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5a740629d0 0x7f5a740c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.042+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5a740629d0 0x7f5a740c7c80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.043+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a740c7420 con 0x7f5a740629d0 2024-05-09T08:56:42.059 INFO:tasks.workunit.client.0.smithi049.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T08:56:42.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.062+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a740c8490 con 0x7f5a740629d0 2024-05-09T08:56:42.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.065+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a740c8490 con 0x7f5a740629d0 2024-05-09T08:56:42.064 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.065+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5a7402c550 0x7f5a7402c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.065+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5a740c8410 con 0x7f5a7402c550 2024-05-09T08:56:42.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.066+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5a7402c550 0x7f5a7402c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.069+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5a7402c550 0x7f5a7402c9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.071+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a740c8410 con 0x7f5a7402c550 2024-05-09T08:56:42.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.105+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a740c81c0 con 0x7f5a7402c550 2024-05-09T08:56:42.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.108+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a740c81c0 con 0x7f5a7402c550 2024-05-09T08:56:42.107 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:42.107 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:42.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.108+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5a74209bf0 0x7f5a7420bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.108+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5a740c8330 con 0x7f5a74209bf0 2024-05-09T08:56:42.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.109+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5a74209bf0 0x7f5a7420bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.111+0000 7f5a737fe640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5a74209bf0 0x7f5a7420bfe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.111+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a740c8330 con 0x7f5a74209bf0 2024-05-09T08:56:42.125 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:42.126+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:42.129 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.129 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.139 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:56:42.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.145+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a740c8330 con 0x7f5a74209bf0 2024-05-09T08:56:42.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.147+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a740c8330 con 0x7f5a74209bf0 2024-05-09T08:56:42.147 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.148+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5a7420ca30 0x7f5a7420ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.148+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5a7402d510 con 0x7f5a7420ca30 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.148+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5a7420ca30 0x7f5a7420ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.149+0000 7f5a78f5e640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5a7420ca30 0x7f5a7420ee20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.149+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a7402d510 con 0x7f5a7420ca30 2024-05-09T08:56:42.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.166+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a7420f360 con 0x7f5a7420ca30 2024-05-09T08:56:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.168+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a7420f360 con 0x7f5a7420ca30 2024-05-09T08:56:42.167 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:42.167 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.168+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5a7420f870 0x7f5a74211c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.168+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5a74214c60 con 0x7f5a7420f870 2024-05-09T08:56:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.168+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5a7420f870 0x7f5a74211c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:42.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.169+0000 7f5a73fff640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5a7420f870 0x7f5a74211c60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:42.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.170+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a74214c60 con 0x7f5a7420f870 2024-05-09T08:56:42.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.175+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f49e0068c70 con 0x7f49e0154c70 2024-05-09T08:56:42.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.177+0000 7f49c7fff640 1 -- 172.21.15.49:0/2904836391 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v20) ==== 81+0+34082 (secure 0 0 0) 0x7f49d40798d0 con 0x7f49e0154c70 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 20, 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "created": "2024-05-09T08:55:44.445697+0000", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "modified": "2024-05-09T08:56:02.682459+0000", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "last_up_change": "2024-05-09T08:55:55.016806+0000", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "last_in_change": "2024-05-09T08:55:46.620125+0000", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_num": 5799936, 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_set": [ 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "pglog_hardlimit", 2024-05-09T08:56:42.177 INFO:teuthology.orchestra.run.smithi049.stdout: "purged_snapdirs", 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_deletes", 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "sortbitwise" 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_version": 4, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "full_ratio": 0.94999998807907104, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_snapshot": "", 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_max": 3, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd": 8, 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "require_min_compat_client": "luminous", 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "min_compat_client": "jewel", 2024-05-09T08:56:42.178 INFO:teuthology.orchestra.run.smithi049.stdout: "require_osd_release": "squid", 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_crimson": false, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "pools": [ 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 1, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": ".mgr", 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:55:56.616182+0000", 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:56:42.179 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 1, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 1, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 1, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 1, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 1, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:56:42.180 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "16", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:56:42.181 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:56:42.182 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:56:42.183 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_max": 32, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 1 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.184 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": {} 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 8, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 8, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 0.25, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 2, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 2, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 1, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 1, 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 1 2024-05-09T08:56:42.185 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 2, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cephfs_metadata", 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:55:59.400583+0000", 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:56:42.186 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 32, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 32, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 32, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 32, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 32, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:56:42.187 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "19", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:56:42.188 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:56:42.189 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:56:42.190 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_bias": 4, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 16, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_priority": 5 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs": { 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": "cephfs" 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.191 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 1.5, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 1.5, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 1, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 1.5, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 1.5, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 1, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 1, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 1 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 3, 2024-05-09T08:56:42.192 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cephfs_data", 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:56:00.024628+0000", 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:56:42.193 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 64, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 64, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 64, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 64, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 64, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.194 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "20", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:56:42.195 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:56:42.196 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:56:42.197 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 64, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "target_size_ratio": 0.89999997615814209 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs": { 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "data": "cephfs" 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 1.5, 2024-05-09T08:56:42.198 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 1.5, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 1, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 1.5, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 1.5, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 1, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 1, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 1 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-05-09T08:56:42.199 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "b59d26bf-6806-4e49-b117-9d83d3a24243", 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6802", 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6803", 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.202 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6806", 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6807", 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.203 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6804", 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6805", 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6801/1154213557", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6803/1154213557", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6807/1154213557", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6805/1154213557", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "d579c6a2-0370-4c21-96fb-6420583dbe74", 2024-05-09T08:56:42.205 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6816", 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6817", 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6818", 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6819", 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.208 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6823", 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6825", 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.209 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6820", 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6821", 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6817/3346547076", 2024-05-09T08:56:42.210 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6819/3346547076", 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6825/3346547076", 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6821/3346547076", 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "bb2ad0e5-2ec1-4647-bc42-3d20911e038a", 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.211 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6808", 2024-05-09T08:56:42.212 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6809", 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.213 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6810", 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6811", 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6814", 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6815", 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.215 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6812", 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6813", 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:56:42.216 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6809/136930210", 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6811/136930210", 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6815/136930210", 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6813/136930210", 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.217 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "546a2935-7144-4c29-b647-e77d89072d60", 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.218 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6822", 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6824", 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6826", 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6827", 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.220 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6830", 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6831", 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6828", 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.222 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6829", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6824/1536339711", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6827/1536339711", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6831/1536339711", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6829/1536339711", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.223 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "f4bb828b-6036-4886-aec7-be53c00ac603", 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.224 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6808", 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6809", 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6810", 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6811", 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6814", 2024-05-09T08:56:42.227 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6815", 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6812", 2024-05-09T08:56:42.228 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6813", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6809/173400960", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6811/173400960", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6815/173400960", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6813/173400960", 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.229 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "41625c33-9f16-4279-b2aa-a63ac0aa6320", 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.230 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6800", 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6801", 2024-05-09T08:56:42.231 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6802", 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6803", 2024-05-09T08:56:42.232 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6806", 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.233 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6807", 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6804", 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6805", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6801/4279483611", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6803/4279483611", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6807/4279483611", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6805/4279483611", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "9177552d-bc69-409e-9654-e030c3a1afba", 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.236 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6824", 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6825", 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.237 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6826", 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6827", 2024-05-09T08:56:42.238 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6830", 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6831", 2024-05-09T08:56:42.239 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6828", 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.240 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6829", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6825/3521264583", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6827/3521264583", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6831/3521264583", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6829/3521264583", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "d3b1821c-1ed4-4286-b7de-030fda235fbf", 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 18, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:56:42.242 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6816", 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6817", 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6818", 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6819", 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.244 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6822", 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6823", 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.245 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6820", 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6821", 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.246 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6817/2743896476", 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6819/2743896476", 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2743896476", 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2743896476", 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_xinfo": [ 2024-05-09T08:56:42.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.763716+0000", 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.248 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.782386+0000", 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.712986+0000", 2024-05-09T08:56:42.249 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.846199+0000", 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-05-09T08:56:42.250 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.735683+0000", 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.251 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.776861+0000", 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.767124+0000", 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.746641+0000", 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap": [], 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_items": [], 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_primaries": [], 2024-05-09T08:56:42.253 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_temp": [], 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_temp": [], 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklist": {}, 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "range_blocklist": {}, 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profiles": { 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "default": { 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-failure-domain": "osd", 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "k": "2", 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "m": "1", 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin": "jerasure", 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "technique": "reed_sol_van" 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:42.254 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps_queue": [], 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "new_removed_snaps": [], 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "new_purged_snaps": [], 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_node_flags": {}, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "device_class_flags": {}, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode": { 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_enabled": false, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_bucket_count": 0, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "degraded_stretch_mode": 0, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "recovering_stretch_mode": 0, 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_bucket": 0 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:42.255 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:42.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 msgr2=0x7f49cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 0x7f49cc06e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009560 tx=0x7f49d0009190 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 msgr2=0x7f49e01559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 0x7f49e01559b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49e0073600 tx=0x7f49d40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 shutdown_connections 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49cc06be30 0x7f49cc06e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49e015b2a0 0x7f49e014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e0154c70 0x7f49e01559b0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49e0073600 tx=0x7f49d40046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 --2- 172.21.15.49:0/2904836391 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e00c8d20 0x7f49e01f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 >> 172.21.15.49:0/2904836391 conn(0x7f49e0074840 msgr2=0x7f49e006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 shutdown_connections 2024-05-09T08:56:42.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.181+0000 7f49e6215640 1 -- 172.21.15.49:0/2904836391 wait complete. 2024-05-09T08:56:42.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.191+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5a74213980 con 0x7f5a7420f870 2024-05-09T08:56:42.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.193+0000 7f5a70ff9640 1 -- 172.21.15.49:0/3648563334 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5a74213980 con 0x7f5a7420f870 2024-05-09T08:56:42.258 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5a741589b0 msgr2=0x7f5a740c4cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5a741589b0 0x7f5a740c4cd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5a74158d90 msgr2=0x7f5a741f8870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5a74158d90 0x7f5a741f8870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f5a741f92c0 msgr2=0x7f5a74061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f5a741f92c0 0x7f5a74061c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5a740629d0 msgr2=0x7f5a740c7c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5a740629d0 0x7f5a740c7c80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5a7402c550 msgr2=0x7f5a7402c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5a7402c550 0x7f5a7402c9b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5a74209bf0 msgr2=0x7f5a7420bfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5a74209bf0 0x7f5a7420bfe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5a7420ca30 msgr2=0x7f5a7420ee20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5a7420ca30 0x7f5a7420ee20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5a7420f870 msgr2=0x7f5a74211c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5a7420f870 0x7f5a74211c60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5a6006be60 msgr2=0x7f5a6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5a6006be60 0x7f5a6006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a64001eb0 tx=0x7f5a64007040 comp rx=0 tx=0).stop 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 msgr2=0x7f5a740c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 --2- 172.21.15.49:0/3648563334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5a741f5d80 0x7f5a740c3f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011af0 tx=0x7f5a6800f240 comp rx=0 tx=0).stop 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a78f5e640 1 -- 172.21.15.49:0/3648563334 reap_dead start 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a78f5e640 1 -- 172.21.15.49:0/3648563334 reap_dead start 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a78f5e640 1 -- 172.21.15.49:0/3648563334 reap_dead start 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a78f5e640 1 -- 172.21.15.49:0/3648563334 reap_dead start 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a78f5e640 1 -- 172.21.15.49:0/3648563334 reap_dead start 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 shutdown_connections 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.194+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 >> 172.21.15.49:0/3648563334 conn(0x7f5a740c9440 msgr2=0x7f5a7415a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.195+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 shutdown_connections 2024-05-09T08:56:42.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.195+0000 7f5a7a9c1640 1 -- 172.21.15.49:0/3648563334 wait complete. 2024-05-09T08:56:42.264 INFO:tasks.thrashosds.thrasher:j is {'epoch': 20, 'fsid': 'aab8b2de-6010-439c-b8fd-8bc72445265c', 'created': '2024-05-09T08:55:44.445697+0000', 'modified': '2024-05-09T08:56:02.682459+0000', 'last_up_change': '2024-05-09T08:55:55.016806+0000', 'last_in_change': '2024-05-09T08:55:46.620125+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:55:59.400583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:56:00.024628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'b59d26bf-6806-4e49-b117-9d83d3a24243', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6800', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6801', 'nonce': 1154213557}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6802', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6803', 'nonce': 1154213557}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6806', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6807', 'nonce': 1154213557}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6804', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6805', 'nonce': 1154213557}]}, 'public_addr': '172.21.15.49:6801/1154213557', 'cluster_addr': '172.21.15.49:6803/1154213557', 'heartbeat_back_addr': '172.21.15.49:6807/1154213557', 'heartbeat_front_addr': '172.21.15.49:6805/1154213557', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd579c6a2-0370-4c21-96fb-6420583dbe74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6816', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6817', 'nonce': 3346547076}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6818', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6819', 'nonce': 3346547076}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6823', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6825', 'nonce': 3346547076}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6820', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6821', 'nonce': 3346547076}]}, 'public_addr': '172.21.15.49:6817/3346547076', 'cluster_addr': '172.21.15.49:6819/3346547076', 'heartbeat_back_addr': '172.21.15.49:6825/3346547076', 'heartbeat_front_addr': '172.21.15.49:6821/3346547076', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bb2ad0e5-2ec1-4647-bc42-3d20911e038a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6808', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6809', 'nonce': 136930210}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6810', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6811', 'nonce': 136930210}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6814', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6815', 'nonce': 136930210}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6812', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6813', 'nonce': 136930210}]}, 'public_addr': '172.21.15.49:6809/136930210', 'cluster_addr': '172.21.15.49:6811/136930210', 'heartbeat_back_addr': '172.21.15.49:6815/136930210', 'heartbeat_front_addr': '172.21.15.49:6813/136930210', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '546a2935-7144-4c29-b647-e77d89072d60', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6822', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6824', 'nonce': 1536339711}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6826', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6827', 'nonce': 1536339711}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6830', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6831', 'nonce': 1536339711}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6828', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6829', 'nonce': 1536339711}]}, 'public_addr': '172.21.15.49:6824/1536339711', 'cluster_addr': '172.21.15.49:6827/1536339711', 'heartbeat_back_addr': '172.21.15.49:6831/1536339711', 'heartbeat_front_addr': '172.21.15.49:6829/1536339711', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f4bb828b-6036-4886-aec7-be53c00ac603', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 173400960}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 173400960}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 173400960}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 173400960}]}, 'public_addr': '172.21.15.183:6809/173400960', 'cluster_addr': '172.21.15.183:6811/173400960', 'heartbeat_back_addr': '172.21.15.183:6815/173400960', 'heartbeat_front_addr': '172.21.15.183:6813/173400960', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '41625c33-9f16-4279-b2aa-a63ac0aa6320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 4279483611}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 4279483611}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 4279483611}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 4279483611}]}, 'public_addr': '172.21.15.183:6801/4279483611', 'cluster_addr': '172.21.15.183:6803/4279483611', 'heartbeat_back_addr': '172.21.15.183:6807/4279483611', 'heartbeat_front_addr': '172.21.15.183:6805/4279483611', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9177552d-bc69-409e-9654-e030c3a1afba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 3521264583}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 3521264583}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 3521264583}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 3521264583}]}, 'public_addr': '172.21.15.183:6825/3521264583', 'cluster_addr': '172.21.15.183:6827/3521264583', 'heartbeat_back_addr': '172.21.15.183:6831/3521264583', 'heartbeat_front_addr': '172.21.15.183:6829/3521264583', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd3b1821c-1ed4-4286-b7de-030fda235fbf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2743896476}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2743896476}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2743896476}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2743896476}]}, 'public_addr': '172.21.15.183:6817/2743896476', 'cluster_addr': '172.21.15.183:6819/2743896476', 'heartbeat_back_addr': '172.21.15.183:6823/2743896476', 'heartbeat_front_addr': '172.21.15.183:6821/2743896476', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.763716+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.782386+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.712986+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.846199+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.735683+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.776861+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.767124+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.746641+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-09T08:56:42.264 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-05-09T08:56:42.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.230+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7eff1c0c8830 con 0x7eff04004800 2024-05-09T08:56:42.265 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:42.257+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:42.265 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.266 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:56:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.280+0000 7eff18ff9640 1 -- 172.21.15.49:0/7180130 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v21) ==== 87+0+0 (secure 0 0 0) 0x7eff0c05d1e0 con 0x7eff04004800 2024-05-09T08:56:42.279 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:56:42.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 msgr2=0x7eff0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 0x7eff0806e320 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff1002f000 tx=0x7eff1002d000 comp rx=0 tx=0).stop 2024-05-09T08:56:42.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 msgr2=0x7eff1c1f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00e7c0 tx=0x7eff0c00ec80 comp rx=0 tx=0).stop 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 shutdown_connections 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff0806be60 0x7eff0806e320 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c1415b0 0x7eff1c070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c1582c0 0x7eff1c1f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 --2- 172.21.15.49:0/7180130 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff04004800 0x7eff1c1f4bb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 >> 172.21.15.49:0/7180130 conn(0x7eff1c0c9440 msgr2=0x7eff1c159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 shutdown_connections 2024-05-09T08:56:42.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:42.283+0000 7eff22370640 1 -- 172.21.15.49:0/7180130 wait complete. 2024-05-09T08:56:42.358 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:42.359+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:42.388 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.389 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:56:42.459 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:42.460+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:42.552 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.552 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.559 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:42.560+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:42.561 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:56:42.661 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:42.662+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:42.718 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.718 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.728 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:56:42.762 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:42.762+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:42.862 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:42.863+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:42.878 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:42.878 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:42.888 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:56:42.963 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:42.964+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:43.039 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.039 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.048 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:56:43.064 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:43.066+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:43.158 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.158 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.165 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:43.166+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:43.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:56:43.266 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:43.267+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:43.278 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.278 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:56:43.372 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:43.373+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:43.439 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.439 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.448 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:56:43.489 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:43.490+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:43.569 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:43.570+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:43.598 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.598 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.607 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:56:43.670 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:43.671+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:43.723 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.723 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.733 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:56:43.777 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:43.778+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:43.836 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.836 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:56:43.872 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:43.872+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:43.973 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:43.974+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:43.988 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:43.988 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:43.997 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:56:44.074 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:44.075+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:44.140 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:44.140 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:44.148 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:56:44.192 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:44.193+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:44.251 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:44.251 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:44.259 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:56:44.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:56:44.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:56:44.310 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:44.311+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:44.377 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:44.378+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.378+0000 7f1fa06c4640 1 Processor -- start 2024-05-09T08:56:44.377 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.378+0000 7f1fa06c4640 1 -- start start 2024-05-09T08:56:44.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1fa06c4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1f98154f70 0x7f1f980c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1fa06c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f980c6b80 0x7f1f98071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1fa06c4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f98072600 con 0x7f1f98154b90 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1fa06c4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1f980728e0 con 0x7f1f98154f70 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1fa06c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1f980c5880 con 0x7f1f980c6b80 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9dc5f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1f98154f70 0x7f1f980c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9e460640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1f98154b90 0x7f1f980cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35740/0 (socket says 172.21.15.49:35740) 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9e460640 1 -- 172.21.15.49:0/3684535906 learned_addr learned my addr 172.21.15.49:0/3684535906 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f980c6b80 0x7f1f98071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9dc5f640 1 -- 172.21.15.49:0/3684535906 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f980c6b80 msgr2=0x7f1f98071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:44.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f980c6b80 0x7f1f98071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9dc5f640 1 -- 172.21.15.49:0/3684535906 >> v1:172.21.15.49:6789/0 conn(0x7f1f98154b90 legacy=0x7f1f980cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9dc5f640 1 -- 172.21.15.49:0/3684535906 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f980c5b60 con 0x7f1f98154f70 2024-05-09T08:56:44.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.379+0000 7f1f9d45e640 1 -- 172.21.15.49:0/3684535906 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257699647 0 0) 0x7f1f98072600 con 0x7f1f98154b90 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.380+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f980c6640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f880097a0 tx=0x7f1f8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71d92cb90ad5a58 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1f9d45e640 1 -- 172.21.15.49:0/3684535906 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88037070 con 0x7f1f98154f70 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1f9d45e640 1 -- 172.21.15.49:0/3684535906 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88032440 con 0x7f1f98154f70 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1f9d45e640 1 -- 172.21.15.49:0/3684535906 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88032810 con 0x7f1f98154f70 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 -- 172.21.15.49:0/3684535906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 msgr2=0x7f1f980c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f980c6640 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f880097a0 tx=0x7f1f8802a210 comp rx=0 tx=0).stop 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 -- 172.21.15.49:0/3684535906 shutdown_connections 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f980c6b80 0x7f1f98071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/3684535906 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f980c6640 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 -- 172.21.15.49:0/3684535906 >> 172.21.15.49:0/3684535906 conn(0x7f1f98074840 msgr2=0x7f1f98074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 -- 172.21.15.49:0/3684535906 shutdown_connections 2024-05-09T08:56:44.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.381+0000 7f1fa06c4640 1 -- 172.21.15.49:0/3684535906 wait complete. 2024-05-09T08:56:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 Processor -- start 2024-05-09T08:56:44.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 -- start start 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f980c6b80 0x7f1f98159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f98154b90 0x7f1f98157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f98149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980c6fa0 con 0x7f1f98154f70 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f980c6120 con 0x7f1f980c6b80 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1fa06c4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9815af40 con 0x7f1f98154b90 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1f9ec61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f98149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1f9ec61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f98149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41004/0 (socket says 172.21.15.49:41004) 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.383+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 learned_addr learned my addr 172.21.15.49:0/2320656992 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f98154b90 0x7f1f98157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f980c6b80 0x7f1f98159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f98154b90 msgr2=0x7f1f98157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f98154b90 0x7f1f98157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f980c6b80 msgr2=0x7f1f98159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f980c6b80 0x7f1f98159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f98150f50 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f98149bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007e90 tx=0x7f1f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90014420 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f90019070 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f981f1560 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90014800 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.384+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f981555c0 con 0x7f1f98154f70 2024-05-09T08:56:44.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.385+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1f9804f660 con 0x7f1f98154f70 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.386+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f900149a0 con 0x7f1f98154f70 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.386+0000 7f1f8effd640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1f7c06be30 0x7f1f7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.386+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f9008b4d0 con 0x7f1f98154f70 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.386+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v21) ==== 54+0+16 (secure 0 0 0) 0x7f1f9008db50 con 0x7f1f98154f70 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.386+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1f7c06be30 0x7f1f7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.387+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1f9815ab60 0x7f1f9814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.387+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f980ca8e0 con 0x7f1f9815ab60 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.387+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1f9815ab60 0x7f1f9814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.387+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1f7c06be30 0x7f1f7c06e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000f60 tx=0x7f1f94007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.387+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1f9815ab60 0x7f1f9814f5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.388+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f980ca8e0 con 0x7f1f9815ab60 2024-05-09T08:56:44.404 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:44.404 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:44.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:56:44.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.420+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f981f1730 con 0x7f1f9815ab60 2024-05-09T08:56:44.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.422+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f981f1730 con 0x7f1f9815ab60 2024-05-09T08:56:44.421 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.422+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1f98150040 0x7f1f981f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.422+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1f981582c0 con 0x7f1f98150040 2024-05-09T08:56:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.422+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1f98150040 0x7f1f981f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.423+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1f98150040 0x7f1f981f3d70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.423+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f981582c0 con 0x7f1f98150040 2024-05-09T08:56:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.433+0000 7f07fe8d4640 1 Processor -- start 2024-05-09T08:56:44.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.433+0000 7f07fe8d4640 1 -- start start 2024-05-09T08:56:44.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fe8d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 0x7f07f8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fe8d4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f8159b10 con 0x7f07f8072670 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fe8d4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f8159270 con 0x7f07f80c7130 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fe8d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f07f8159550 con 0x7f07f8152e30 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fce71640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f07f8072670 0x7f07f80c3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35748/0 (socket says 172.21.15.49:35748) 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07fce71640 1 -- 172.21.15.49:0/2353890144 learned_addr learned my addr 172.21.15.49:0/2353890144 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.434+0000 7f07f7fff640 1 --2- 172.21.15.49:0/2353890144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 0x7f07f8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104493193 0 0) 0x7f07f8159b10 con 0x7f07f8072670 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07e0003520 con 0x7f07f8072670 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3539029424 0 0) 0x7f07f8159270 con 0x7f07f80c7130 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f8159b10 con 0x7f07f80c7130 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f7fff640 1 -- 172.21.15.49:0/2353890144 >> v1:172.21.15.183:6789/0 conn(0x7f07f80c7130 legacy=0x7f07f80c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f7fff640 1 -- 172.21.15.49:0/2353890144 >> v1:172.21.15.49:6789/0 conn(0x7f07f8072670 legacy=0x7f07f80c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.435+0000 7f07f7fff640 1 -- 172.21.15.49:0/2353890144 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f8159830 con 0x7f07f8152e30 2024-05-09T08:56:44.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07f7fff640 1 --2- 172.21.15.49:0/2353890144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 0x7f07f8158be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097d0 tx=0x7f07e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c583321aba93663f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8037070 con 0x7f07f8152e30 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e8032440 con 0x7f07f8152e30 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07f6ffd640 1 -- 172.21.15.49:0/2353890144 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8032810 con 0x7f07f8152e30 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 -- 172.21.15.49:0/2353890144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 msgr2=0x7f07f8158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/2353890144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 0x7f07f8158be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f07e80097d0 tx=0x7f07e802a210 comp rx=0 tx=0).stop 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 -- 172.21.15.49:0/2353890144 shutdown_connections 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/2353890144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f8152e30 0x7f07f8158be0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 -- 172.21.15.49:0/2353890144 >> 172.21.15.49:0/2353890144 conn(0x7f07f80cc290 msgr2=0x7f07f80cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:44.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 -- 172.21.15.49:0/2353890144 shutdown_connections 2024-05-09T08:56:44.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.436+0000 7f07fe8d4640 1 -- 172.21.15.49:0/2353890144 wait complete. 2024-05-09T08:56:44.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.437+0000 7f07fe8d4640 1 Processor -- start 2024-05-09T08:56:44.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 -- start start 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 0x7f07f81e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07f8152e30 0x7f07f81e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80c6620 con 0x7f07f8072670 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07f80c59f0 con 0x7f07f8152e30 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.438+0000 7f07fe8d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f07f806b4b0 con 0x7f07f80c7130 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41018/0 (socket says 172.21.15.49:41018) 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f7fff640 1 -- 172.21.15.49:0/3719166527 learned_addr learned my addr 172.21.15.49:0/3719166527 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 0x7f07f81e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07fce71640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07f8152e30 0x7f07f81e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 -- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07f8152e30 msgr2=0x7f07f81e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07f8152e30 0x7f07f81e21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 -- 172.21.15.49:0/3719166527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 msgr2=0x7f07f814bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 -- 172.21.15.49:0/3719166527 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f81e0bf0 con 0x7f07f80c7130 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f7fff640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f77fe640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 0x7f07f81e1c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00c3c0 tx=0x7f07ec00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec017070 con 0x7f07f80c7130 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec0052f0 con 0x7f07f80c7130 2024-05-09T08:56:44.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81e0ed0 con 0x7f07f80c7130 2024-05-09T08:56:44.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec003830 con 0x7f07f80c7130 2024-05-09T08:56:44.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.439+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f81e1490 con 0x7f07f80c7130 2024-05-09T08:56:44.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.440+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f980cc030 con 0x7f1f98150040 2024-05-09T08:56:44.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.440+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f804f660 con 0x7f07f80c7130 2024-05-09T08:56:44.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.441+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07ec01a070 con 0x7f07f80c7130 2024-05-09T08:56:44.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.441+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f980cc030 con 0x7f1f98150040 2024-05-09T08:56:44.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.441+0000 7f07f4ff9640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 0x7f07e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.446 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:44.447 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:44.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f07ec08a100 con 0x7f07f80c7130 2024-05-09T08:56:44.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1f98061c70 0x7f1f98062050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1f981f4b80 con 0x7f1f98061c70 2024-05-09T08:56:44.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1f98061c70 0x7f1f98062050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1f98061c70 0x7f1f98062050 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.442+0000 7f07f7fff640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 0x7f07e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.443+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f981f4b80 con 0x7f1f98061c70 2024-05-09T08:56:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.444+0000 7f07f7fff640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 0x7f07e406e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07e8000c00 tx=0x7f07e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.446+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f07ec058490 con 0x7f07f80c7130 2024-05-09T08:56:44.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.470+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f980627e0 con 0x7f1f98061c70 2024-05-09T08:56:44.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.471+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f980627e0 con 0x7f1f98061c70 2024-05-09T08:56:44.471 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.472+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1f9802c550 0x7f1f9802c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.472+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1f980cfaa0 con 0x7f1f9802c550 2024-05-09T08:56:44.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.472+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1f9802c550 0x7f1f9802c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.472+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1f9802c550 0x7f1f9802c9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.473+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f980cfaa0 con 0x7f1f9802c550 2024-05-09T08:56:44.477 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:44.478+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:44.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.488+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f9802d1c0 con 0x7f1f9802c550 2024-05-09T08:56:44.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.490+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f9802d1c0 con 0x7f1f9802c550 2024-05-09T08:56:44.489 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:44.489 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:44.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.490+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1f980ce8a0 0x7f1f980cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.490+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1f9802d3d0 con 0x7f1f980ce8a0 2024-05-09T08:56:44.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.491+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1f980ce8a0 0x7f1f980cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.493+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1f980ce8a0 0x7f1f980cec80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.494+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f9802d3d0 con 0x7f1f980ce8a0 2024-05-09T08:56:44.516 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:44.516 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:44.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.525+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f9802d3d0 con 0x7f1f980ce8a0 2024-05-09T08:56:44.525 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:56:44.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.527+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f9802d3d0 con 0x7f1f980ce8a0 2024-05-09T08:56:44.526 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.527+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1f980cf6d0 0x7f1f98207100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.527+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1f98209ed0 con 0x7f1f980cf6d0 2024-05-09T08:56:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.527+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1f980cf6d0 0x7f1f98207100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.528+0000 7f1f9dc5f640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1f980cf6d0 0x7f1f98207100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.529+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f98209ed0 con 0x7f1f980cf6d0 2024-05-09T08:56:44.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.543+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f98207640 con 0x7f1f980cf6d0 2024-05-09T08:56:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.545+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f98207640 con 0x7f1f980cf6d0 2024-05-09T08:56:44.544 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:44.544 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.545+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1f98207b50 0x7f1f98209f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.545+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1f9820cd10 con 0x7f1f98207b50 2024-05-09T08:56:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.545+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1f98207b50 0x7f1f98209f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.546+0000 7f1f9ec61640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1f98207b50 0x7f1f98209f40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.546+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f9820cd10 con 0x7f1f98207b50 2024-05-09T08:56:44.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.561+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f9820a480 con 0x7f1f98207b50 2024-05-09T08:56:44.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.563+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f9820a480 con 0x7f1f98207b50 2024-05-09T08:56:44.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.563+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1f9820ab00 0x7f1f9820cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.563+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1f9820fe00 con 0x7f1f9820ab00 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.563+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1f9820ab00 0x7f1f9820cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.564+0000 7f1f9e460640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1f9820ab00 0x7f1f9820cef0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:44.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.564+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f9820fe00 con 0x7f1f9820ab00 2024-05-09T08:56:44.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.579+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1f9820d430 con 0x7f1f9820ab00 2024-05-09T08:56:44.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1f8effd640 1 -- 172.21.15.49:0/2320656992 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1f9820d430 con 0x7f1f9820ab00 2024-05-09T08:56:44.580 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1f9815ab60 msgr2=0x7f1f9814f5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1f9815ab60 0x7f1f9814f5f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1f98150040 msgr2=0x7f1f981f3d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1f98150040 0x7f1f981f3d70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1f98061c70 msgr2=0x7f1f98062050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1f98061c70 0x7f1f98062050 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1f9802c550 msgr2=0x7f1f9802c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1f9802c550 0x7f1f9802c9b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 reap_dead start 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1f980ce8a0 msgr2=0x7f1f980cec80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1f980ce8a0 0x7f1f980cec80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 reap_dead start 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1f980cf6d0 msgr2=0x7f1f98207100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1f980cf6d0 0x7f1f98207100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1f98207b50 msgr2=0x7f1f98209f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1f98207b50 0x7f1f98209f40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1f9820ab00 msgr2=0x7f1f9820cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1f9820ab00 0x7f1f9820cef0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1f7c06be30 msgr2=0x7f1f7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1f7c06be30 0x7f1f7c06e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f94000f60 tx=0x7f1f94007d50 comp rx=0 tx=0).stop 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 msgr2=0x7f1f98149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 --2- 172.21.15.49:0/2320656992 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1f98154f70 0x7f1f98149bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f90007e90 tx=0x7f1f9000e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1f9ec61640 1 -- 172.21.15.49:0/2320656992 reap_dead start 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 shutdown_connections 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 >> 172.21.15.49:0/2320656992 conn(0x7f1f98074840 msgr2=0x7f1f980d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 shutdown_connections 2024-05-09T08:56:44.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.581+0000 7f1fa06c4640 1 -- 172.21.15.49:0/2320656992 wait complete. 2024-05-09T08:56:44.610 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:44.611+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:44.669 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:44.669 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:44.669 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:44.670 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:44.670 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:44.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:44.677+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f07f814f690 con 0x7f07f80c7130 2024-05-09T08:56:44.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:56:44.719 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:44.720+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:44.780 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:44.781+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.142063176999997, 2024-05-09T08:56:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:56:44.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:56:44.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.142060309999998, 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.829 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:56:44.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:56:44.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.142056664999998, 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:56:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:56:44.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:56:44.839 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.099247499999997, 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:56:44.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:56:44.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:44.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.845 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.094455511, 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:56:44.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:56:44.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.091949143000001, 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:56:44.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:44.855 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.090477752999998, 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:56:44.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:56:44.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 27.090401915000001, 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.860 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:56:44.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:56:44.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077580505, 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:56:44.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:56:44.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:56:44.870 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077559507, 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:56:44.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077555112999999, 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.876 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:56:44.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.881 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077550055, 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:56:44.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:56:44.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077546573999999, 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:56:44.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:56:44.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077541956000001, 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:56:44.893 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:56:44.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:56:44.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077538487000002, 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:56:44.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077535185999999, 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:56:44.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:56:44.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:56:44.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.077222117000002, 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:56:44.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.061146295, 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.915 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:56:44.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.920 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.061141616, 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:56:44.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:44.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.061135598, 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:56:44.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:44.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:56:44.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:44.932 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:44.934 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:56:44.934 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:44.934+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:45.035 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:45.036+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:45.076 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:45.076 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:45.076 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:45.076 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:45.085 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:56:45.169 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:45.170+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:45.228 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:45.228 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:45.228 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:45.228 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:45.228 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:45.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:56:45.237 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:45.238+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:45.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.291+0000 7f07f4ff9640 1 -- 172.21.15.49:0/3719166527 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v22) ==== 97+0+0 (secure 0 0 0) 0x7f07ec05c790 con 0x7f07f80c7130 2024-05-09T08:56:45.290 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 msgr2=0x7f07e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 0x7f07e406e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07e8000c00 tx=0x7f07e802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 msgr2=0x7f07f81e1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 0x7f07f81e1c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00c3c0 tx=0x7f07ec00c880 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 shutdown_connections 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f07e406be60 0x7f07e406e320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07f8152e30 0x7f07f81e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07f80c7130 0x7f07f81e1c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 --2- 172.21.15.49:0/3719166527 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f07f8072670 0x7f07f814bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:45.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 >> 172.21.15.49:0/3719166527 conn(0x7f07f80cc290 msgr2=0x7f07f8151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:45.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 shutdown_connections 2024-05-09T08:56:45.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:45.293+0000 7f07fe8d4640 1 -- 172.21.15.49:0/3719166527 wait complete. 2024-05-09T08:56:45.337 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:45.338+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.719998979000003, 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:56:45.384 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:56:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:56:45.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:56:45.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.391 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327342715, 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.392 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:56:45.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:56:45.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:56:45.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327295122999999, 2024-05-09T08:56:45.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327245255999998, 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:56:45.407 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:56:45.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:56:45.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:56:45.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327239456000001, 2024-05-09T08:56:45.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:56:45.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:56:45.421 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327127656000002, 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:56:45.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:56:45.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327030493999999, 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:56:45.428 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:56:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:56:45.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:56:45.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.327024735999998, 2024-05-09T08:56:45.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:56:45.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.442 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.326951096000002, 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:56:45.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:56:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.326899284, 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.449 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:56:45.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:56:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:56:45.455 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.326895565000001, 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:56:45.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:56:45.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.841839108000002, 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:56:45.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:56:45.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.841822624000002, 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.470 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:56:45.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.841740667000003, 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:56:45.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:56:45.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:56:45.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.841693106999998, 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:56:45.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:56:45.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:56:45.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.638064663000002, 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:56:45.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:56:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.621985218000001, 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:56:45.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:56:45.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:56:45.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.621980902000001, 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.502 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:56:45.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:56:45.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.621971939, 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:56:45.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.513 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 15.621967879, 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:45.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:45.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:45.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:45.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:45.520 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:45.522 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:45.523+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:45.522 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:56:45.623 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:45.624+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:45.638 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:45.638 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:45.638 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:45.638 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:45.647 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:56:45.731 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:45.732+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:45.797 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:45.797 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:45.797 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:45.797 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:45.797 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:45.806 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:56:45.847 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:45.848+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:45.926 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:45.927+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:45.958 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:45.958 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.277586829000001, 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:56:45.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:56:45.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.277581259999998, 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:56:45.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:45.970 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.277549571999998, 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:56:45.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.976 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.277544886000001, 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:56:45.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:56:45.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.277541057999997, 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:56:45.982 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:56:45.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.276658191999999, 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:56:45.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:56:45.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.991 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 34.231085307000001, 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:56:45.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:56:45.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:45.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:45.997 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 34.230412532000003, 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:45.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:45.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:56:46.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.227691865000001, 2024-05-09T08:56:46.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:56:46.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:56:46.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:56:46.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.008 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.227683824, 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:46.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:56:46.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:56:46.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.227679504000001, 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:56:46.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:56:46.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:56:46.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.227675199, 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:56:46.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.026 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.227671951000001, 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:56:46.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:56:46.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.212599577999999, 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:56:46.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.212588330999999, 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:56:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:56:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.212583909999999, 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:56:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:56:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.196479945, 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:56:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.196468614, 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:56:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.196460664, 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.196454786, 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:56:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.071 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:46.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:56:46.073 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:46.074+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:46.172 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:46.173+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:46.223 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:46.223 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:46.224 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:46.224 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:46.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:56:46.273 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:46.275+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:46.374 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:46.375+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:46.382 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:46.382 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:46.382 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:46.382 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:46.382 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:46.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:56:46.475 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:46.476+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:46.543 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:46.543 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:46.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:46.543 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:46.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.862615374999997, 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:46.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:56:46.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:56:46.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.549 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.862591023, 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:56:46.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:56:46.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:46.555 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.862585267999997, 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:56:46.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:56:46.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.862570036999998, 2024-05-09T08:56:46.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:46.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:56:46.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:56:46.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 34.819232051999997, 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:56:46.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:56:46.572 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 34.819224091000002, 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:46.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:56:46.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.812016062000001, 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:56:46.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.812005317000001, 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:56:46.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:56:46.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.798030743999998, 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:56:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:56:46.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:46.596 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.798020635, 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:56:46.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:56:46.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.798017996999999, 2024-05-09T08:56:46.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:46.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:56:46.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:56:46.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.798002395000001, 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:56:46.608 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:56:46.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:56:46.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.797999745999999, 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:56:46.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:56:46.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.797985619999999, 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:46.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:56:46.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.797983340999998, 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:56:46.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:56:46.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.797975436000002, 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:56:46.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.635 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.797973915, 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:56:46.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:56:46.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.777898065999999, 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:56:46.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:56:46.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.777876606, 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:56:46.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:56:46.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:46.651 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 16.777869600999999, 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:56:46.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:46.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:46.656 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:46.656 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:56:46.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:56:46.658 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:46.659+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:46.758 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:46.759+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:46.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.802+0000 7f424e9e3640 1 Processor -- start 2024-05-09T08:56:46.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.802+0000 7f424e9e3640 1 -- start start 2024-05-09T08:56:46.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424e9e3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4248154b90 0x7f42480cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424e9e3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4248072600 con 0x7f42480c6d50 2024-05-09T08:56:46.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424e9e3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42480728e0 con 0x7f4248154f70 2024-05-09T08:56:46.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424e9e3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f42480c5880 con 0x7f4248154b90 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f4247fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4248154b90 0x7f42480cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424cf80640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f42480c6d50 0x7f4248071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35752/0 (socket says 172.21.15.49:35752) 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f424cf80640 1 -- 172.21.15.49:0/2913736662 learned_addr learned my addr 172.21.15.49:0/2913736662 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f4247fff640 1 -- 172.21.15.49:0/2913736662 >> v1:172.21.15.183:6789/0 conn(0x7f4248154f70 legacy=0x7f42480c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f4247fff640 1 -- 172.21.15.49:0/2913736662 >> v1:172.21.15.49:6789/0 conn(0x7f42480c6d50 legacy=0x7f4248071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f4247fff640 1 -- 172.21.15.49:0/2913736662 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480c5b60 con 0x7f4248154b90 2024-05-09T08:56:46.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.803+0000 7f4246ffd640 1 -- 172.21.15.49:0/2913736662 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4186616567 0 0) 0x7f4248072600 con 0x7f42480c6d50 2024-05-09T08:56:46.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.804+0000 7f4247fff640 1 --2- 172.21.15.49:0/2913736662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154b90 0x7f42480cb280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42380097a0 tx=0x7f423802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c35aecd802a82e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.804+0000 7f4246ffd640 1 -- 172.21.15.49:0/2913736662 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238037070 con 0x7f4248154b90 2024-05-09T08:56:46.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.804+0000 7f4246ffd640 1 -- 172.21.15.49:0/2913736662 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4238032640 con 0x7f4248154b90 2024-05-09T08:56:46.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.804+0000 7f4246ffd640 1 -- 172.21.15.49:0/2913736662 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238032a10 con 0x7f4248154b90 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.804+0000 7f424e9e3640 1 -- 172.21.15.49:0/2913736662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154b90 msgr2=0x7f42480cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 --2- 172.21.15.49:0/2913736662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154b90 0x7f42480cb280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42380097a0 tx=0x7f423802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 -- 172.21.15.49:0/2913736662 shutdown_connections 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 --2- 172.21.15.49:0/2913736662 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154b90 0x7f42480cb280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 -- 172.21.15.49:0/2913736662 >> 172.21.15.49:0/2913736662 conn(0x7f4248074840 msgr2=0x7f4248074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 -- 172.21.15.49:0/2913736662 shutdown_connections 2024-05-09T08:56:46.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.805+0000 7f424e9e3640 1 -- 172.21.15.49:0/2913736662 wait complete. 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.806+0000 7f424e9e3640 1 Processor -- start 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.806+0000 7f424e9e3640 1 -- start start 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42480c6d50 0x7f42480b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4248154b90 0x7f42480b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 0x7f42480b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f424815af40 con 0x7f4248154f70 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42480c6120 con 0x7f42480c6d50 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424e9e3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f42480cb7c0 con 0x7f4248154b90 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 0x7f42480b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 0x7f42480b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41028/0 (socket says 172.21.15.49:41028) 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 learned_addr learned my addr 172.21.15.49:0/32587347 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42480c6d50 0x7f42480b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4248154b90 0x7f42480b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4248154b90 msgr2=0x7f42480b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4248154b90 0x7f42480b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42480c6d50 msgr2=0x7f42480b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42480c6d50 0x7f42480b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480ce680 con 0x7f4248154f70 2024-05-09T08:56:46.808 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.807+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 0x7f42480b4b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f424000c7d0 tx=0x7f424000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240018070 con 0x7f4248154f70 2024-05-09T08:56:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424000a640 con 0x7f4248154f70 2024-05-09T08:56:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424000aa70 con 0x7f4248154f70 2024-05-09T08:56:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42480cf350 con 0x7f4248154f70 2024-05-09T08:56:46.809 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42480cf910 con 0x7f4248154f70 2024-05-09T08:56:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.808+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f42480cd8f0 con 0x7f4248154f70 2024-05-09T08:56:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.809+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f424001b070 con 0x7f4248154f70 2024-05-09T08:56:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.810+0000 7f4244ff9640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f423406be60 0x7f423406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.810+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f424008a8b0 con 0x7f4248154f70 2024-05-09T08:56:46.810 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.810+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v22) ==== 54+0+16 (secure 0 0 0) 0x7f42480cd8f0 con 0x7f4248154f70 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.810+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f423406be60 0x7f423406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.811+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f42480cfb50 0x7f4248071820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.811+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42480ce430 con 0x7f42480cfb50 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.811+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f42480cfb50 0x7f4248071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.811+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f42480cfb50 0x7f4248071820 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.811+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f423406be60 0x7f423406e320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4238034000 tx=0x7f423802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.812+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42480ce430 con 0x7f42480cfb50 2024-05-09T08:56:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:46.812 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:46.812 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:46.812 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:46.821 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:56:46.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.827+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424804f660 con 0x7f42480cfb50 2024-05-09T08:56:46.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.829+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424804f660 con 0x7f42480cfb50 2024-05-09T08:56:46.828 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.829+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f42480ca4e0 0x7f424814f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.829+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f42481542b0 con 0x7f42480ca4e0 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.829+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f42480ca4e0 0x7f424814f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.829+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f42480ca4e0 0x7f424814f5f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.830+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42481542b0 con 0x7f42480ca4e0 2024-05-09T08:56:46.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.845+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42480ca9e0 con 0x7f42480ca4e0 2024-05-09T08:56:46.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.847+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42480ca9e0 con 0x7f42480ca4e0 2024-05-09T08:56:46.846 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:46.847 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:46.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.847+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f42481f49f0 0x7f42481f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.847+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f42480cbaf0 con 0x7f42481f49f0 2024-05-09T08:56:46.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.847+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f42481f49f0 0x7f42481f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.848+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f42481f49f0 0x7f42481f4e30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.848+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42480cbaf0 con 0x7f42481f49f0 2024-05-09T08:56:46.858 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:46.859+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:46.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.863+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4248061f40 con 0x7f42481f49f0 2024-05-09T08:56:46.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.864+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4248061f40 con 0x7f42481f49f0 2024-05-09T08:56:46.864 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:46.864 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.865+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f42480cbde0 0x7f42480cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.865+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f42480cd220 con 0x7f42480cbde0 2024-05-09T08:56:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.865+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f42480cbde0 0x7f42480cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.865+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f42480cbde0 0x7f42480cc240 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.866+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42480cd220 con 0x7f42480cbde0 2024-05-09T08:56:46.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.880+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424815b730 con 0x7f42480cbde0 2024-05-09T08:56:46.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.882+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424815b730 con 0x7f42480cbde0 2024-05-09T08:56:46.881 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.882+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f42480ccc90 0x7f424802c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.882+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f424815b730 con 0x7f42480ccc90 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.882+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f42480ccc90 0x7f424802c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.883+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f42480ccc90 0x7f424802c040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.883+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f424815b730 con 0x7f42480ccc90 2024-05-09T08:56:46.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.898+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424815b730 con 0x7f42480ccc90 2024-05-09T08:56:46.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.900+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424815b730 con 0x7f42480ccc90 2024-05-09T08:56:46.899 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.900+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f424802ca90 0x7f424802ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.900+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f424802d3f0 con 0x7f424802ca90 2024-05-09T08:56:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.900+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f424802ca90 0x7f424802ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.901+0000 7f42477fe640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f424802ca90 0x7f424802ceb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.901 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.902+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f424802d3f0 con 0x7f424802ca90 2024-05-09T08:56:46.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.916+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424802d3f0 con 0x7f424802ca90 2024-05-09T08:56:46.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.918+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424802d3f0 con 0x7f424802ca90 2024-05-09T08:56:46.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.918+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4248207d80 0x7f424820a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.918+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f424820cf40 con 0x7f4248207d80 2024-05-09T08:56:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.918+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4248207d80 0x7f424820a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.919+0000 7f424cf80640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4248207d80 0x7f424820a170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.920+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f424820cf40 con 0x7f4248207d80 2024-05-09T08:56:46.931 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:46.932 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:46.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.934+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424820a6b0 con 0x7f4248207d80 2024-05-09T08:56:46.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.935+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424820a6b0 con 0x7f4248207d80 2024-05-09T08:56:46.935 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:46.935 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:46.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.936+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f424820ad30 0x7f424820d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:46.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.936+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f424820ff70 con 0x7f424820ad30 2024-05-09T08:56:46.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.936+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f424820ad30 0x7f424820d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:46.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.936+0000 7f4247fff640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f424820ad30 0x7f424820d120 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:46.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.937+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f424820ff70 con 0x7f424820ad30 2024-05-09T08:56:46.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:56:46.950 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.951+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f424820d930 con 0x7f424820ad30 2024-05-09T08:56:46.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f4244ff9640 1 -- 172.21.15.49:0/32587347 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f424820d930 con 0x7f424820ad30 2024-05-09T08:56:46.953 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f42480cfb50 msgr2=0x7f4248071820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f42480cfb50 0x7f4248071820 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f42480ca4e0 msgr2=0x7f424814f5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f42480ca4e0 0x7f424814f5f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f42481f49f0 msgr2=0x7f42481f4e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f42481f49f0 0x7f42481f4e30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f42480cbde0 msgr2=0x7f42480cc240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f42480cbde0 0x7f42480cc240 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 reap_dead start 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f42480ccc90 msgr2=0x7f424802c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f42480ccc90 0x7f424802c040 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 reap_dead start 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f424802ca90 msgr2=0x7f424802ceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f424802ca90 0x7f424802ceb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4248207d80 msgr2=0x7f424820a170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4248207d80 0x7f424820a170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f424820ad30 msgr2=0x7f424820d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f424820ad30 0x7f424820d120 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f423406be60 msgr2=0x7f423406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f423406be60 0x7f423406e320 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4238034000 tx=0x7f423802e000 comp rx=0 tx=0).stop 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 msgr2=0x7f42480b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 --2- 172.21.15.49:0/32587347 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4248154f70 0x7f42480b4b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f424000c7d0 tx=0x7f424000cc90 comp rx=0 tx=0).stop 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 reap_dead start 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424cf80640 1 -- 172.21.15.49:0/32587347 reap_dead start 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 shutdown_connections 2024-05-09T08:56:46.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.954+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 >> 172.21.15.49:0/32587347 conn(0x7f4248074840 msgr2=0x7f424814a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.955+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 shutdown_connections 2024-05-09T08:56:46.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:46.955+0000 7f424e9e3640 1 -- 172.21.15.49:0/32587347 wait complete. 2024-05-09T08:56:46.959 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:46.960+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:47.060 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:47.061+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:47.060 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.061 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:56:47.161 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:47.162+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:47.223 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.223 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:56:47.265 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:56:47.265 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:56:47.265 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-09T08:56:47.265 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.781699 2024-05-09T08:56:47.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.7816992685098567 2024-05-09T08:56:47.276 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:47.277+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:47.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:56:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.357 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.362 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:47.363+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:47.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:56:47.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.394+0000 7f7d6dedf640 1 Processor -- start 2024-05-09T08:56:47.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.394+0000 7f7d6dedf640 1 -- start start 2024-05-09T08:56:47.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d6dedf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d680c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d6dedf640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7d680c6b80 0x7f7d68071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d6dedf640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d680c6fd0 con 0x7f7d68154b90 2024-05-09T08:56:47.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d6dedf640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7d68072600 con 0x7f7d680c6b80 2024-05-09T08:56:47.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d6dedf640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7d680728e0 con 0x7f7d68154f70 2024-05-09T08:56:47.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d677fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7d68154b90 0x7f7d680cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:35760/0 (socket says 172.21.15.49:35760) 2024-05-09T08:56:47.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d677fe640 1 -- 172.21.15.49:0/2208618685 learned_addr learned my addr 172.21.15.49:0/2208618685 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:47.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d67fff640 1 --2- 172.21.15.49:0/2208618685 >> v2:172.21.15.183:3300/0 conn(0x7f7d680c6b80 0x7f7d68071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d680c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2561636912 0 0) 0x7f7d680c6fd0 con 0x7f7d68154b90 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d50003520 con 0x7f7d68154b90 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2062432512 0 0) 0x7f7d50003520 con 0x7f7d68154b90 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 msgr2=0x7f7d680c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d680c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 >> v2:172.21.15.183:3300/0 conn(0x7f7d680c6b80 msgr2=0x7f7d68071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 --2- 172.21.15.49:0/2208618685 >> v2:172.21.15.183:3300/0 conn(0x7f7d680c6b80 0x7f7d68071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d680c5880 con 0x7f7d68154b90 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f7d58001670 con 0x7f7d68154b90 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.395+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 >> v1:172.21.15.49:6789/0 conn(0x7f7d68154b90 legacy=0x7f7d680cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d500039b0 0x7f7d50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d50004d40 0x7f7d500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68072600 con 0x7f7d50004d40 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680728e0 con 0x7f7d500039b0 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5000b9c0 con 0x7f7d68154f70 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d50004d40 0x7f7d500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d677fe640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d500039b0 0x7f7d50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 -- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 msgr2=0x7f7d50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 -- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d500039b0 msgr2=0x7f7d50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d500039b0 0x7f7d50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d67fff640 1 -- 172.21.15.49:0/2208618685 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d50003520 con 0x7f7d50004d40 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d677fe640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d500039b0 0x7f7d50003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:47.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.396+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d68154f70 0x7f7d50004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:47.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.397+0000 7f7d67fff640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d50004d40 0x7f7d500051c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009fc0 tx=0x7f7d50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb78c1d79c1caffe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.397+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60009c10 con 0x7f7d50004d40 2024-05-09T08:56:47.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.397+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60036070 con 0x7f7d50004d40 2024-05-09T08:56:47.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.397+0000 7f7d667fc640 1 -- 172.21.15.49:0/2208618685 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6002d070 con 0x7f7d50004d40 2024-05-09T08:56:47.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 -- 172.21.15.49:0/2208618685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d50004d40 msgr2=0x7f7d500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/2208618685 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d50004d40 0x7f7d500051c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009fc0 tx=0x7f7d50005830 comp rx=0 tx=0).stop 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d67fff640 1 -- 172.21.15.49:0/2208618685 reap_dead start 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 -- 172.21.15.49:0/2208618685 shutdown_connections 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 -- 172.21.15.49:0/2208618685 >> 172.21.15.49:0/2208618685 conn(0x7f7d68074840 msgr2=0x7f7d68074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 -- 172.21.15.49:0/2208618685 shutdown_connections 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.398+0000 7f7d6dedf640 1 -- 172.21.15.49:0/2208618685 wait complete. 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.399+0000 7f7d6dedf640 1 Processor -- start 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 -- start start 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d680b5c70 0x7f7d681522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d68152810 0x7f7d681f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680c6120 con 0x7f7d680b5890 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680c5e40 con 0x7f7d68152810 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d6dedf640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6815af40 con 0x7f7d680b5c70 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d677fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41046/0 (socket says 172.21.15.49:41046) 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.400+0000 7f7d677fe640 1 -- 172.21.15.49:0/1233052812 learned_addr learned my addr 172.21.15.49:0/1233052812 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d67fff640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d68152810 0x7f7d681f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d680b5c70 0x7f7d681522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 -- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d680b5c70 msgr2=0x7f7d681522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d680b5c70 0x7f7d681522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 -- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d68152810 msgr2=0x7f7d681f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d68152810 0x7f7d681f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 -- 172.21.15.49:0/1233052812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d68152d10 con 0x7f7d680b5890 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d677fe640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d58007fd0 tx=0x7f7d5800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58019070 con 0x7f7d680b5890 2024-05-09T08:56:47.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d58004cb0 con 0x7f7d680b5890 2024-05-09T08:56:47.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d68152ff0 con 0x7f7d680b5890 2024-05-09T08:56:47.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004e60 con 0x7f7d680b5890 2024-05-09T08:56:47.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.401+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d681535b0 con 0x7f7d680b5890 2024-05-09T08:56:47.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.403+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6804f660 con 0x7f7d680b5890 2024-05-09T08:56:47.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.403+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d5801c070 con 0x7f7d680b5890 2024-05-09T08:56:47.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.404+0000 7f7d4ffff640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.404+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7d5808aa40 con 0x7f7d680b5890 2024-05-09T08:56:47.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.404+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.405+0000 7f7d66ffd640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d6815aca0 tx=0x7f7d5c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.408+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7d58058d10 con 0x7f7d680b5890 2024-05-09T08:56:47.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.428+0000 7f8e7c189640 1 Processor -- start 2024-05-09T08:56:47.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.428+0000 7f8e7c189640 1 -- start start 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e7c189640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e74154b90 0x7f8e740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e7c189640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154f70 0x7f8e740cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e7c189640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e74072600 con 0x7f8e740c6640 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e7c189640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8e740728e0 con 0x7f8e74154b90 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e7c189640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e740c5880 con 0x7f8e74154f70 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e79f25640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e74154b90 0x7f8e740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e79f25640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e74154b90 0x7f8e740cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41052/0 (socket says 172.21.15.49:41052) 2024-05-09T08:56:47.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e79f25640 1 -- 172.21.15.49:0/715460075 learned_addr learned my addr 172.21.15.49:0/715460075 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:47.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.429+0000 7f8e79724640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154f70 0x7f8e740cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.430+0000 7f8e79f25640 1 -- 172.21.15.49:0/715460075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154f70 msgr2=0x7f8e740cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.430+0000 7f8e79f25640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154f70 0x7f8e740cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.430+0000 7f8e79f25640 1 -- 172.21.15.49:0/715460075 >> v1:172.21.15.183:6789/0 conn(0x7f8e740c6640 legacy=0x7f8e740c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:47.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.430+0000 7f8e79f25640 1 -- 172.21.15.49:0/715460075 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e740c5b60 con 0x7f8e74154b90 2024-05-09T08:56:47.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.430+0000 7f8e79f25640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154b90 0x7f8e740cb280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009870 tx=0x7f8e7002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b90a204dab8b88e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e78f23640 1 -- 172.21.15.49:0/715460075 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70037070 con 0x7f8e74154b90 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e78f23640 1 -- 172.21.15.49:0/715460075 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7002ae50 con 0x7f8e74154b90 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e78f23640 1 -- 172.21.15.49:0/715460075 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70032860 con 0x7f8e74154b90 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 -- 172.21.15.49:0/715460075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154b90 msgr2=0x7f8e740cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154b90 0x7f8e740cb280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009870 tx=0x7f8e7002a300 comp rx=0 tx=0).stop 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 -- 172.21.15.49:0/715460075 shutdown_connections 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154f70 0x7f8e740cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 --2- 172.21.15.49:0/715460075 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154b90 0x7f8e740cb280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 -- 172.21.15.49:0/715460075 >> 172.21.15.49:0/715460075 conn(0x7f8e74074840 msgr2=0x7f8e74074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 -- 172.21.15.49:0/715460075 shutdown_connections 2024-05-09T08:56:47.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.431+0000 7f8e7c189640 1 -- 172.21.15.49:0/715460075 wait complete. 2024-05-09T08:56:47.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.432+0000 7f8e7c189640 1 Processor -- start 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 -- start start 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e740c6640 0x7f8e740cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154b90 0x7f8e740ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7415af40 con 0x7f8e74154f70 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e740c6120 con 0x7f8e740c6640 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7c189640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e74071820 con 0x7f8e74154b90 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7a726640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7a726640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41062/0 (socket says 172.21.15.49:41062) 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.433+0000 7f8e7a726640 1 -- 172.21.15.49:0/1953314897 learned_addr learned my addr 172.21.15.49:0/1953314897 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e79724640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154b90 0x7f8e740ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e79f25640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e740c6640 0x7f8e740cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 -- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154b90 msgr2=0x7f8e740ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154b90 0x7f8e740ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 -- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e740c6640 msgr2=0x7f8e740cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e740c6640 0x7f8e740cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 -- 172.21.15.49:0/1953314897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e740d0140 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7a726640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c7d0 tx=0x7f8e6c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003b70 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6c003d30 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7414c890 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c00aa90 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.434+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e740ca070 con 0x7f8e74154f70 2024-05-09T08:56:47.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.435+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e7404f660 con 0x7f8e74154f70 2024-05-09T08:56:47.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.436+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e6c00ac30 con 0x7f8e74154f70 2024-05-09T08:56:47.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.437+0000 7f8e6a7fc640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:47.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.437+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8e6c08ac20 con 0x7f8e74154f70 2024-05-09T08:56:47.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.438+0000 7f8e79f25640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:47.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.438+0000 7f8e79f25640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 0x7f8e5806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e70034000 tx=0x7f8e7002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:47.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.442+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e6c058f80 con 0x7f8e74154f70 2024-05-09T08:56:47.463 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:47.465+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:47.508 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.508 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.517 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:56:47.564 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:47.565+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.608+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 0.7816992685098567} v 0) -- 0x7f7d680cd290 con 0x7f7d680b5890 2024-05-09T08:56:47.631 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.632 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:47.636+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f8e74158920 con 0x7f8e74154f70 2024-05-09T08:56:47.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:56:47.665 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:47.666+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:47.744 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.744 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.753 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:56:47.765 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:47.766+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:47.855 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.855 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:56:47.866 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:47.867+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:47.966 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:47.967 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:47.967 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:47.968+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:47.975 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:56:48.068 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:48.069+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:48.119 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.119 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:56:48.169 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:48.170+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:48.270 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.271 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.271 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:48.272+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:48.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:56:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.294+0000 7f7d4ffff640 1 -- 172.21.15.49:0/1233052812 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 0.7816992685098567}]=0 set osd.2 primary-affinity to 0.781699 (8512292) v23) ==== 155+0+0 (secure 0 0 0) 0x7f7d5808ae20 con 0x7f7d680b5890 2024-05-09T08:56:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.294+0000 7f8e6a7fc640 1 -- 172.21.15.49:0/1953314897 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v23) ==== 91+0+0 (secure 0 0 0) 0x7f8e6c05d280 con 0x7f8e74154f70 2024-05-09T08:56:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.2 primary-affinity to 0.781699 (8512292) 2024-05-09T08:56:48.294 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:56:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.297+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 msgr2=0x7f7d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.297+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 0x7f7d5406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d6815aca0 tx=0x7f7d5c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.297+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 msgr2=0x7f7d681f09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:48.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.297+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7d58007fd0 tx=0x7f7d5800dd40 comp rx=0 tx=0).stop 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 shutdown_connections 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7d5406be30 0x7f7d5406e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d68152810 0x7f7d681f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7d680b5c70 0x7f7d681522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 --2- 172.21.15.49:0/1233052812 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7d680b5890 0x7f7d681f09b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 >> 172.21.15.49:0/1233052812 conn(0x7f7d68074840 msgr2=0x7f7d680cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:48.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 shutdown_connections 2024-05-09T08:56:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 msgr2=0x7f8e5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 0x7f8e5806e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8e70034000 tx=0x7f8e7002e000 comp rx=0 tx=0).stop 2024-05-09T08:56:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 msgr2=0x7f8e7414ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c7d0 tx=0x7f8e6c00cc90 comp rx=0 tx=0).stop 2024-05-09T08:56:48.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f7d6dedf640 1 -- 172.21.15.49:0/1233052812 wait complete. 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 shutdown_connections 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e5806be30 0x7f8e5806e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e74154f70 0x7f8e7414ae60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e74154b90 0x7f8e740ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 --2- 172.21.15.49:0/1953314897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e740c6640 0x7f8e740cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 >> 172.21.15.49:0/1953314897 conn(0x7f8e74074840 msgr2=0x7f8e740cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 shutdown_connections 2024-05-09T08:56:48.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:48.298+0000 7f8e7c189640 1 -- 172.21.15.49:0/1953314897 wait complete. 2024-05-09T08:56:48.371 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:48.372+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:48.385 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.385 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:56:48.478 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:48.479+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:48.536 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.536 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.544 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:56:48.586 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:48.587+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:48.674 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:48.675+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:48.688 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.688 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:56:48.775 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:48.776+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:48.841 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.841 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.849 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:56:48.875 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:48.877+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:48.954 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:48.954 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:48.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:56:48.965 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:56:49.009 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:49.010+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:49.064 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:49.064 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:49.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:56:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.081+0000 7f60baa2a640 1 Processor -- start 2024-05-09T08:56:49.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.081+0000 7f60baa2a640 1 -- start start 2024-05-09T08:56:49.080 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:49.081+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60baa2a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60baa2a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 0x7f60b406f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60baa2a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b415c700 con 0x7f60b41551d0 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60baa2a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f60b4071ef0 con 0x7f60b4154df0 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60baa2a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f60b415ed50 con 0x7f60b40c7e80 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b8fc7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 0x7f60b406f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b37fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f60b41551d0 0x7f60b415e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:56226/0 (socket says 172.21.15.49:56226) 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41070/0 (socket says 172.21.15.49:41070) 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b37fe640 1 -- 172.21.15.49:0/1095795833 learned_addr learned my addr 172.21.15.49:0/1095795833 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b8fc7640 1 -- 172.21.15.49:0/1095795833 >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 msgr2=0x7f60b415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/1095795833 >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b2ffd640 1 -- 172.21.15.49:0/1095795833 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236739699 0 0) 0x7f60b415c700 con 0x7f60b41551d0 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b8fc7640 1 -- 172.21.15.49:0/1095795833 >> v1:172.21.15.183:6789/0 conn(0x7f60b41551d0 legacy=0x7f60b415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b8fc7640 1 -- 172.21.15.49:0/1095795833 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b415f030 con 0x7f60b40c7e80 2024-05-09T08:56:49.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.082+0000 7f60b3fff640 1 --2- 172.21.15.49:0/1095795833 >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/1095795833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 0x7f60b406f210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009d90 tx=0x7f60ac02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70813396a2ffbf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60b2ffd640 1 -- 172.21.15.49:0/1095795833 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac007740 con 0x7f60b40c7e80 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60b2ffd640 1 -- 172.21.15.49:0/1095795833 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac0078e0 con 0x7f60b40c7e80 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60b2ffd640 1 -- 172.21.15.49:0/1095795833 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac007cb0 con 0x7f60b40c7e80 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 -- 172.21.15.49:0/1095795833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 msgr2=0x7f60b406f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 --2- 172.21.15.49:0/1095795833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 0x7f60b406f210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009d90 tx=0x7f60ac02a660 comp rx=0 tx=0).stop 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 -- 172.21.15.49:0/1095795833 shutdown_connections 2024-05-09T08:56:49.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 --2- 172.21.15.49:0/1095795833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b40c7e80 0x7f60b406f210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 --2- 172.21.15.49:0/1095795833 >> v2:172.21.15.49:3300/0 conn(0x7f60b4154df0 0x7f60b415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 -- 172.21.15.49:0/1095795833 >> 172.21.15.49:0/1095795833 conn(0x7f60b406b200 msgr2=0x7f60b406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.084+0000 7f60baa2a640 1 -- 172.21.15.49:0/1095795833 shutdown_connections 2024-05-09T08:56:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.085+0000 7f60baa2a640 1 -- 172.21.15.49:0/1095795833 wait complete. 2024-05-09T08:56:49.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.086+0000 7f60baa2a640 1 Processor -- start 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.086+0000 7f60baa2a640 1 -- start start 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 0x7f60b415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b4154df0 0x7f60b40c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60b414cf80 0x7f60b414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f60b415b480 con 0x7f60b40c7e80 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60b415b1a0 con 0x7f60b414cf80 2024-05-09T08:56:49.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60baa2a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f60b415e9d0 con 0x7f60b4154df0 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 0x7f60b415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 0x7f60b415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41078/0 (socket says 172.21.15.49:41078) 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 -- 172.21.15.49:0/3502697523 learned_addr learned my addr 172.21.15.49:0/3502697523 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b4154df0 0x7f60b40c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60b414cf80 0x7f60b414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b4154df0 msgr2=0x7f60b40c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b4154df0 0x7f60b40c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60b414cf80 msgr2=0x7f60b414d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60b414cf80 0x7f60b414d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b41f03f0 con 0x7f60b40c7e80 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60b4154df0 0x7f60b40c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.087+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 0x7f60b415d6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f60a400ec10 tx=0x7f60a400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.088+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4017070 con 0x7f60b40c7e80 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.088+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a400cc30 con 0x7f60b40c7e80 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.088+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b40ca650 con 0x7f60b40c7e80 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.088+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4013470 con 0x7f60b40c7e80 2024-05-09T08:56:49.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.088+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b415e370 con 0x7f60b40c7e80 2024-05-09T08:56:49.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.089+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f60b404f660 con 0x7f60b40c7e80 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.089+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60a401a070 con 0x7f60b40c7e80 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60b0ff9640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f60a006be30 0x7f60a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5741+0+0 (secure 0 0 0) 0x7f60a408b1e0 con 0x7f60b40c7e80 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v23) ==== 54+0+16 (secure 0 0 0) 0x7f60b41f03f0 con 0x7f60b40c7e80 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f60a006be30 0x7f60a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f60b41551d0 0x7f60b41f0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60b414f2a0 con 0x7f60b41551d0 2024-05-09T08:56:49.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.090+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f60b41551d0 0x7f60b41f0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.091+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f60a006be30 0x7f60a006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f60b40c4690 tx=0x7f60a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.091+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f60b41551d0 0x7f60b41f0670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.092+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b414f2a0 con 0x7f60b41551d0 2024-05-09T08:56:49.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.113+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b40c8400 con 0x7f60b41551d0 2024-05-09T08:56:49.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.114+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b40c8400 con 0x7f60b41551d0 2024-05-09T08:56:49.114 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:49.114 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:49.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.114+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f60b41f3e70 0x7f60b41f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.114+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f60b41f4790 con 0x7f60b41f3e70 2024-05-09T08:56:49.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.115+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f60b41f3e70 0x7f60b41f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.115+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f60b41f3e70 0x7f60b41f4250 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.116+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b41f4790 con 0x7f60b41f3e70 2024-05-09T08:56:49.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.131+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b415df30 con 0x7f60b41f3e70 2024-05-09T08:56:49.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.133+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b415df30 con 0x7f60b41f3e70 2024-05-09T08:56:49.132 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.133+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f60b41480e0 0x7f60b41484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.133+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f60b4148a00 con 0x7f60b41480e0 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.133+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f60b41480e0 0x7f60b41484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.133+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f60b41480e0 0x7f60b41484c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.134+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b4148a00 con 0x7f60b41480e0 2024-05-09T08:56:49.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.149+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b4061f40 con 0x7f60b41480e0 2024-05-09T08:56:49.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.151+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b4061f40 con 0x7f60b41480e0 2024-05-09T08:56:49.150 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.151+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f60b4158920 0x7f60b4158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.151+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f60b4062a20 con 0x7f60b4158920 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.151+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f60b4158920 0x7f60b4158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.151+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f60b4158920 0x7f60b4158d60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.151 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.152+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b4062a20 con 0x7f60b4158920 2024-05-09T08:56:49.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.166+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b4159570 con 0x7f60b4158920 2024-05-09T08:56:49.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.168+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b4159570 con 0x7f60b4158920 2024-05-09T08:56:49.168 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.168+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f60b402c550 0x7f60b402c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.168+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f60b4062c20 con 0x7f60b402c550 2024-05-09T08:56:49.168 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.169+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f60b402c550 0x7f60b402c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.169+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f60b402c550 0x7f60b402c990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.169 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.170+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b4062c20 con 0x7f60b402c550 2024-05-09T08:56:49.177 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:49.178+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:49.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.184+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b4062c20 con 0x7f60b402c550 2024-05-09T08:56:49.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.186+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b4062c20 con 0x7f60b402c550 2024-05-09T08:56:49.185 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:49.186 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:49.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.186+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f60b4204f60 0x7f60b4207350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.186+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f60b402d3e0 con 0x7f60b4204f60 2024-05-09T08:56:49.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.186+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f60b4204f60 0x7f60b4207350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.187+0000 7f60b37fe640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f60b4204f60 0x7f60b4207350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.187+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b402d3e0 con 0x7f60b4204f60 2024-05-09T08:56:49.192 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:49.193 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:49.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:56:49.202 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.202+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b402d3e0 con 0x7f60b4204f60 2024-05-09T08:56:49.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.204+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b402d3e0 con 0x7f60b4204f60 2024-05-09T08:56:49.203 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:49.204 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:49.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.204+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f60b4207da0 0x7f60b420a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.204+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f60b41597d0 con 0x7f60b4207da0 2024-05-09T08:56:49.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.204+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f60b4207da0 0x7f60b420a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.205+0000 7f60b8fc7640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f60b4207da0 0x7f60b420a190 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.205+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b41597d0 con 0x7f60b4207da0 2024-05-09T08:56:49.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.220+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b420a6d0 con 0x7f60b4207da0 2024-05-09T08:56:49.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.222+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b420a6d0 con 0x7f60b4207da0 2024-05-09T08:56:49.221 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:49.222 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:49.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.222+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f60b420abe0 0x7f60b420cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:49.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.222+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f60b420ff60 con 0x7f60b420abe0 2024-05-09T08:56:49.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.222+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f60b420abe0 0x7f60b420cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:49.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.223+0000 7f60b3fff640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f60b420abe0 0x7f60b420cfd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:49.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.223+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60b420ff60 con 0x7f60b420abe0 2024-05-09T08:56:49.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.238+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f60b420eca0 con 0x7f60b420abe0 2024-05-09T08:56:49.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60b0ff9640 1 -- 172.21.15.49:0/3502697523 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f60b420eca0 con 0x7f60b420abe0 2024-05-09T08:56:49.240 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f60b41551d0 msgr2=0x7f60b41f0670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f60b41551d0 0x7f60b41f0670 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f60b41f3e70 msgr2=0x7f60b41f4250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f60b41f3e70 0x7f60b41f4250 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f60b41480e0 msgr2=0x7f60b41484c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.240+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f60b41480e0 0x7f60b41484c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f60b4158920 msgr2=0x7f60b4158d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f60b4158920 0x7f60b4158d60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60b8fc7640 1 -- 172.21.15.49:0/3502697523 reap_dead start 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f60b402c550 msgr2=0x7f60b402c990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f60b402c550 0x7f60b402c990 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60b8fc7640 1 -- 172.21.15.49:0/3502697523 reap_dead start 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f60b4204f60 msgr2=0x7f60b4207350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f60b4204f60 0x7f60b4207350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f60b4207da0 msgr2=0x7f60b420a190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f60b4207da0 0x7f60b420a190 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f60b420abe0 msgr2=0x7f60b420cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f60b420abe0 0x7f60b420cfd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f60a006be30 msgr2=0x7f60a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f60a006be30 0x7f60a006e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f60b40c4690 tx=0x7f60a8009040 comp rx=0 tx=0).stop 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 msgr2=0x7f60b415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 --2- 172.21.15.49:0/3502697523 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f60b40c7e80 0x7f60b415d6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f60a400ec10 tx=0x7f60a400c520 comp rx=0 tx=0).stop 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60b8fc7640 1 -- 172.21.15.49:0/3502697523 reap_dead start 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 shutdown_connections 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 >> 172.21.15.49:0/3502697523 conn(0x7f60b406b200 msgr2=0x7f60b4150240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 shutdown_connections 2024-05-09T08:56:49.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:49.241+0000 7f60baa2a640 1 -- 172.21.15.49:0/3502697523 wait complete. 2024-05-09T08:56:49.278 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:49.279+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:49.316 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:49.317 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:49.317 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:49.317 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:49.317 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:49.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:56:49.378 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:49.379+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:49.437 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:49.437 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:49.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.757156797, 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:56:49.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:56:49.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:56:49.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:56:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.757151227999998, 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:49.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:56:49.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.757119539999998, 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:56:49.454 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:56:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:56:49.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:56:49.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.461 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.757114854000001, 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:56:49.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:49.467 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.757111025999997, 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:49.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:56:49.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:49.472 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.756228159999999, 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:56:49.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.710655275000001, 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.480 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:56:49.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:56:49.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.488 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.709982500000002, 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:56:49.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 31.707261833, 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:56:49.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:56:49.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 31.707253791999999, 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:49.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:56:49.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.506 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 31.707249472000001, 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:56:49.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:56:49.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:56:49.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:56:49.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 31.707245167, 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:56:49.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:56:49.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:56:49.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 31.707241919000001, 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:56:49.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 25.692169545999999, 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:56:49.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 25.692158298999999, 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:56:49.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 25.692153877999999, 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:56:49.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:56:49.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.676049913, 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.542 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:56:49.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.676038582, 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:56:49.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:49.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.553 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.676030632, 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:56:49.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:56:49.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:49.559 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.676024754, 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:56:49.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:56:49.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.563 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:49.564 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:49.564 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:49.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:56:49.568 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:49.568+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:49.668 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:49.669+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:49.720 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:49.720 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:49.721 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:49.721 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:49.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:56:49.770 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:49.771+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:49.871 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:49.872+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:49.880 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:49.881 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:49.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:56:49.975 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:49.976+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.361314034000003, 2024-05-09T08:56:50.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:56:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.361289681999999, 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:56:50.048 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:56:50.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:56:50.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.361283927000002, 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:56:50.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:56:50.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:56:50.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.059 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.361268696000003, 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:56:50.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:56:50.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:56:50.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 38.317930711000002, 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:56:50.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:56:50.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:56:50.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 38.317922750000001, 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:56:50.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:56:50.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.077 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.310714720999997, 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:56:50.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:56:50.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.310703975999999, 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:56:50.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:56:50.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296729403000001, 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:56:50.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:56:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296719293999999, 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.096 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:56:50.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:56:50.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296716656000001, 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:56:50.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:56:50.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296701054, 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:50.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:56:50.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:56:50.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:56:50.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296698405000001, 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.114 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:56:50.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:56:50.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:56:50.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296684279000001, 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.120 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:56:50.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296682000000001, 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:56:50.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:56:50.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296674095, 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:56:50.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.136 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 26.296672573999999, 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:50.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:56:50.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 20.276596725000001, 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.141 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:56:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 20.276575265000002, 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:56:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:56:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:56:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 20.276568260000001, 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:56:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.157 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:50.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:56:50.159 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:50.160+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:50.260 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:50.261+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:50.311 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:50.312 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:50.312 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:50.312 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:50.317 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:56:50.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:56:50.361 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:50.362+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:50.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.423+0000 7fd37a73b640 1 Processor -- start 2024-05-09T08:56:50.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.423+0000 7fd37a73b640 1 -- start start 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd37a73b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd374154b90 0x7fd3740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd37a73b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd374154f70 0x7fd3740cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd37a73b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd374072600 con 0x7fd3740c6640 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd37a73b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd3740728e0 con 0x7fd374154f70 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd37a73b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd3740c5880 con 0x7fd374154b90 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.424+0000 7fd36bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd374154f70 0x7fd3740cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd374154f70 0x7fd3740cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41084/0 (socket says 172.21.15.49:41084) 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 -- 172.21.15.49:0/3101561122 learned_addr learned my addr 172.21.15.49:0/3101561122 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd373fff640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd374154b90 0x7fd3740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 -- 172.21.15.49:0/3101561122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd374154b90 msgr2=0x7fd3740cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:50.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd374154b90 0x7fd3740cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:50.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 -- 172.21.15.49:0/3101561122 >> v1:172.21.15.183:6789/0 conn(0x7fd3740c6640 legacy=0x7fd3740c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:50.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.425+0000 7fd36bfff640 1 -- 172.21.15.49:0/3101561122 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3740c5b60 con 0x7fd374154f70 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.426+0000 7fd36bfff640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3740cb7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f225b6af4ea0a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.426+0000 7fd3737fe640 1 -- 172.21.15.49:0/3101561122 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c037070 con 0x7fd374154f70 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.426+0000 7fd3737fe640 1 -- 172.21.15.49:0/3101561122 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c02ae50 con 0x7fd374154f70 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.426+0000 7fd3737fe640 1 -- 172.21.15.49:0/3101561122 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c032860 con 0x7fd374154f70 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 -- 172.21.15.49:0/3101561122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 msgr2=0x7fd3740cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3740cb7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009870 tx=0x7fd35c02a300 comp rx=0 tx=0).stop 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 -- 172.21.15.49:0/3101561122 shutdown_connections 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3740cb7c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:50.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 --2- 172.21.15.49:0/3101561122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd374154b90 0x7fd3740cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:50.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 -- 172.21.15.49:0/3101561122 >> 172.21.15.49:0/3101561122 conn(0x7fd374074840 msgr2=0x7fd374074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:50.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 -- 172.21.15.49:0/3101561122 shutdown_connections 2024-05-09T08:56:50.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.427+0000 7fd37a73b640 1 -- 172.21.15.49:0/3101561122 wait complete. 2024-05-09T08:56:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.428+0000 7fd37a73b640 1 Processor -- start 2024-05-09T08:56:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.428+0000 7fd37a73b640 1 -- start start 2024-05-09T08:56:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3740c6640 0x7fd374159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 0x7fd3741e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd37415af40 con 0x7fd374154f70 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3740c6120 con 0x7fd374154b90 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd37a73b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd374071820 con 0x7fd3740c6640 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd378cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd378cd8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41098/0 (socket says 172.21.15.49:41098) 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd378cd8640 1 -- 172.21.15.49:0/2926519483 learned_addr learned my addr 172.21.15.49:0/2926519483 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.429+0000 7fd373fff640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3740c6640 0x7fd374159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd36bfff640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 0x7fd3741e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 -- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3740c6640 msgr2=0x7fd374159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3740c6640 0x7fd374159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:50.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 -- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 msgr2=0x7fd3741e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 0x7fd3741e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 -- 172.21.15.49:0/2926519483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741e10f0 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd36bfff640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 0x7fd3741e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd378cd8640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c7d0 tx=0x7fd36c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c003b70 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd36c003d30 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741f19e0 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c00aa90 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.430+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd374159c70 con 0x7fd374154f70 2024-05-09T08:56:50.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.431+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374151f70 con 0x7fd374154f70 2024-05-09T08:56:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.432+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd36c00ac30 con 0x7fd374154f70 2024-05-09T08:56:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.432+0000 7fd3717fa640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 0x7fd35406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.432+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5857+0+0 (secure 0 0 0) 0x7fd36c08aea0 con 0x7fd374154f70 2024-05-09T08:56:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.433+0000 7fd373fff640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 0x7fd35406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:50.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.433+0000 7fd373fff640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 0x7fd35406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd364009ad0 tx=0x7fd364008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:50.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.437+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd36c059720 con 0x7fd374154f70 2024-05-09T08:56:50.461 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:50.463+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:50.472 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:50.473 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:50.473 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:50.473 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:50.473 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:50.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:56:50.563 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:50.564+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:50.594 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:50.594 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:50.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:50.594 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:50.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913149804, 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005543915, 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680886+0000", 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680905+0000", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9360000000000001e-05 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681012+0000", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107077 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681023+0000", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1236000000000001e-05 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116939 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681272+0000", 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013183499999999999 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681397+0000", 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124456 2024-05-09T08:56:50.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685651+0000", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042543770000000002 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686328+0000", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067728299999999999 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686380+0000", 2024-05-09T08:56:50.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2012000000000003e-05 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686425+0000", 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4305999999999999e-05 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.600 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913120657, 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060063249999999999, 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680914+0000", 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7799999999999998e-06 2024-05-09T08:56:50.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681059+0000", 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000144986 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681066+0000", 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4910000000000002e-06 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682105+0000", 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010389920000000001 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682185+0000", 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0357000000000003e-05 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682323+0000", 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828000000000001 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685656+0000", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033325149999999999 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686805+0000", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001149529 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686877+0000", 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1303000000000005e-05 2024-05-09T08:56:50.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686916+0000", 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9461e-05 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:50.606 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913114880999998, 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006182725, 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:50.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.96e-07 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.61e-06 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00020153400000000001 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681129+0000", 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2620000000000004e-06 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682565+0000", 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014361650000000001 2024-05-09T08:56:50.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682695+0000", 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012924400000000001 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682808+0000", 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113004 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685660+0000", 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002852121 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686999+0000", 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001338802 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687050+0000", 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1625000000000001e-05 2024-05-09T08:56:50.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687098+0000", 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8062000000000002e-05 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913111012999998, 2024-05-09T08:56:50.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063447829999999997, 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680920+0000", 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9200000000000002e-07 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680945+0000", 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4856000000000001e-05 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681153+0000", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000208385 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681161+0000", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3189999999999995e-06 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683100+0000", 2024-05-09T08:56:50.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019385590000000001 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683175+0000", 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4972e-05 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683297+0000", 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012193699999999999 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685664+0000", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023667929999999998 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687171+0000", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001507216 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687223+0000", 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2571999999999997e-05 2024-05-09T08:56:50.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687264+0000", 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0881999999999999e-05 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:50.618 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913083847999999, 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064597120000000003, 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1600000000000001e-07 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680950+0000", 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:56:50.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226572 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681193+0000", 2024-05-09T08:56:50.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6884e-05 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683542+0000", 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023484650000000001 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683657+0000", 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115277 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683766+0000", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010891399999999999 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685668+0000", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001901743 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687313+0000", 2024-05-09T08:56:50.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0016458709999999999 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687362+0000", 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8625999999999998e-05 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687406+0000", 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4276000000000001e-05 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913079386, 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066291869999999999, 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4799999999999999e-07 2024-05-09T08:56:50.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680954+0000", 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6730000000000001e-06 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681306+0000", 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000351491 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681313+0000", 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9619999999999996e-06 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684111+0000", 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027984569999999999 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684202+0000", 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1195999999999999e-05 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684353+0000", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015071299999999999 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686447+0000", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020943440000000002 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687488+0000", 2024-05-09T08:56:50.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001040336 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687534+0000", 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6774999999999999e-05 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687580+0000", 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5892e-05 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913075290999998, 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006791755, 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680956+0000", 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.34e-07 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680958+0000", 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-06 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681332+0000", 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00037391099999999999 2024-05-09T08:56:50.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681338+0000", 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2639999999999999e-06 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684606+0000", 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032684599999999999 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684723+0000", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011661499999999999 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684853+0000", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013001199999999999 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686475+0000", 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001621883 2024-05-09T08:56:50.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687653+0000", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001178438 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687705+0000", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1805000000000003e-05 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687747+0000", 2024-05-09T08:56:50.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2143e-05 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913070937000001, 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069219210000000002, 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.637 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6399999999999998e-07 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680968+0000", 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3289999999999994e-06 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681351+0000", 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00038298100000000001 2024-05-09T08:56:50.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681356+0000", 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.101e-06 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685141+0000", 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003785086 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685240+0000", 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8267e-05 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685406+0000", 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000166644 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686653+0000", 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012463190000000001 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687796+0000", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011430450000000001 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687849+0000", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2923999999999997e-05 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687881+0000", 2024-05-09T08:56:50.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2861000000000003e-05 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.913046465999997, 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048537759999999998, 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.643 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:50.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680987+0000", 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4370000000000001e-06 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681414+0000", 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00042719999999999998 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681421+0000", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.601e-06 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682060+0000", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00063917100000000001 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682139+0000", 2024-05-09T08:56:50.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8872e-05 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682269+0000", 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013019400000000001 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684249+0000", 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001979794 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685731+0000", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014824930000000001 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685788+0000", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6986e-05 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685838+0000", 2024-05-09T08:56:50.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9755000000000001e-05 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:20 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 38.870073636000001, 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012308524, 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723958+0000", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.296e-06 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.395e-06 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723990+0000", 2024-05-09T08:56:50.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6914999999999999e-05 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723999+0000", 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5129999999999996e-06 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724055+0000", 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6159000000000002e-05 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724166+0000", 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110532 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.736265+0000", 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012099713999999999 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.653 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:15 2.3 e20/17 2:c27f454c:::mds1_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 38.868048209000001, 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061558979999999999, 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725983+0000", 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8000000000000004e-07 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725991+0000", 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0860000000000002e-06 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726006+0000", 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4938e-05 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2429999999999999e-06 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727846+0000", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018329609999999999 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727871+0000", 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4406999999999999e-05 2024-05-09T08:56:50.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732095+0000", 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042245640000000001 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732138+0000", 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818999999999998e-05 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.658 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863967471999999, 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057959939999999996, 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:50.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1199999999999998e-07 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730066+0000", 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.694e-06 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730250+0000", 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018380499999999999 2024-05-09T08:56:50.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730260+0000", 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7100000000000002e-06 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731230+0000", 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097010499999999997 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731303+0000", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3579999999999997e-05 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731420+0000", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116568 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732988+0000", 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015678440000000001 2024-05-09T08:56:50.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735786+0000", 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027978769999999998 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1066000000000003e-05 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735859+0000", 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2332999999999997e-05 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863963073999997, 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048474440000000002, 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730068+0000", 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730070+0000", 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.553e-06 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730295+0000", 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00022485300000000001 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730301+0000", 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9580000000000004e-06 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730352+0000", 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1400999999999998e-05 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730483+0000", 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013079300000000001 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010618799999999999 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733630+0000", 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030401299999999998 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734823+0000", 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011937950000000001 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734873+0000", 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9707999999999999e-05 2024-05-09T08:56:50.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734915+0000", 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1746999999999999e-05 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863959051999998, 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050848680000000002, 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730072+0000", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6100000000000002e-07 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730075+0000", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8490000000000002e-06 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730966+0000", 2024-05-09T08:56:50.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089175099999999998 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730979+0000", 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2847000000000001e-05 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731802+0000", 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082315399999999999 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731879+0000", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7046999999999996e-05 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732012+0000", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000133005 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733927+0000", 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001914612 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735049+0000", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011219789999999999 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735113+0000", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4362999999999993e-05 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735156+0000", 2024-05-09T08:56:50.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2899999999999999e-05 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863951610999997, 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.004901702, 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:56:50.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730081+0000", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8220000000000001e-06 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731051+0000", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096957499999999997 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731057+0000", 2024-05-09T08:56:50.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9560000000000002e-06 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731629+0000", 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057226399999999995 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731714+0000", 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5661999999999994e-05 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732235+0000", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052096200000000003 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733613+0000", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013775339999999999 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734929+0000", 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001316198 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734954+0000", 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4477999999999998e-05 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734981+0000", 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6933000000000001e-05 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.684 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863948375, 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053901399999999999, 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8700000000000002e-07 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730085+0000", 2024-05-09T08:56:50.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7609999999999999e-06 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731070+0000", 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098512399999999998 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731083+0000", 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3033999999999999e-05 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732979+0000", 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001895276 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733089+0000", 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110542 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733211+0000", 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121471 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734283+0000", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001071904 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735386+0000", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011033620000000001 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735437+0000", 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0735999999999997e-05 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735472+0000", 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5642999999999997e-05 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863944267000001, 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054919699999999997, 2024-05-09T08:56:50.690 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730087+0000", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7599999999999997e-07 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730089+0000", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731097+0000", 2024-05-09T08:56:50.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001008412 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731102+0000", 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5589999999999999e-06 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732441+0000", 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00133946 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732543+0000", 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010173199999999999 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732613+0000", 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9788000000000003e-05 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734007+0000", 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013943499999999999 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735476+0000", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014689709999999999 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735536+0000", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0408999999999997e-05 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735578+0000", 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1897000000000002e-05 2024-05-09T08:56:50.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863940675000002, 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059167780000000001, 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5699999999999999e-07 2024-05-09T08:56:50.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730096+0000", 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0599999999999996e-06 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731114+0000", 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001017525 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731118+0000", 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7639999999999996e-06 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731633+0000", 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00051442600000000001 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731741+0000", 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010863599999999999 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731906+0000", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016487199999999999 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733654+0000", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001747738 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735909+0000", 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022545 2024-05-09T08:56:50.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735950+0000", 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1201000000000001e-05 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.736007+0000", 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6799000000000003e-05 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863926812999999, 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055945150000000004, 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000002e-07 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730113+0000", 2024-05-09T08:56:50.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1020000000000003e-06 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731131+0000", 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010174279999999999 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731135+0000", 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4449999999999998e-06 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733092+0000", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019572190000000001 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733229+0000", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013633499999999999 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733345+0000", 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116647 2024-05-09T08:56:50.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734293+0000", 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094760700000000005 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735618+0000", 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013248260000000001 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735667+0000", 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9468000000000001e-05 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735698+0000", 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1053000000000002e-05 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:50.707 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.863915988999999, 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057439600000000002, 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:50.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730115+0000", 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730117+0000", 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7940000000000001e-06 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731147+0000", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010300610000000001 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731151+0000", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7269999999999998e-06 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733642+0000", 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024900780000000002 2024-05-09T08:56:50.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733758+0000", 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116214 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735858+0000", 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021007040000000001 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:50.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:50.712 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:50.713 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:56:50.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:50.631+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fd37404f660 con 0x7fd374154f70 2024-05-09T08:56:50.714 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:50.714+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:50.814 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:50.814+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:50.864 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:50.864 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:50.864 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:50.865 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:50.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:56:50.914 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:50.916+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:51.015 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:51.015+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:51.023 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:51.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:51.023 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:51.023 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:51.023 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:51.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:56:51.116 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:51.117+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.502904647000001, 2024-05-09T08:56:51.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:56:51.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.502884676000001, 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:51.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:56:51.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.502055910000003, 2024-05-09T08:56:51.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:51.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:56:51.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.501053018, 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:56:51.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:56:51.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:56:51.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:56:51.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:56:51.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.500750187000001, 2024-05-09T08:56:51.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:51.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:56:51.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.459563629000002, 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.209 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:56:51.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:51.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.459551603000001, 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:56:51.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.459538133000002, 2024-05-09T08:56:51.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.458513730999996, 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:56:51.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.458086442000003, 2024-05-09T08:56:51.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:56:51.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:56:51.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.457970101999997, 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:56:51.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.456696106000003, 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:51.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:56:51.240 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.454946706000001, 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:56:51.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:56:51.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.454618562, 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:56:51.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:56:51.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.444489934000003, 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:51.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:56:51.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:56:51.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:51.255 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.453746932999998, 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:51.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:56:51.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.453737547000003, 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.261 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:56:51.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.267 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.453726502999999, 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:56:51.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.451991507000002, 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:51.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:56:51.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 21.422087301000001, 2024-05-09T08:56:51.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:51.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:56:51.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:56:51.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:51.282 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:51.283 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:56:51.284 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:51.284+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:51.285 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:56:51.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.309+0000 7fd3717fa640 1 -- 172.21.15.49:0/2926519483 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v26) ==== 101+0+0 (secure 0 0 0) 0x7fd36c059020 con 0x7fd374154f70 2024-05-09T08:56:51.310 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.312+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 msgr2=0x7fd35406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.312+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 0x7fd35406e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd364009ad0 tx=0x7fd364008040 comp rx=0 tx=0).stop 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.312+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 msgr2=0x7fd3741e1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.312+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c7d0 tx=0x7fd36c00cc90 comp rx=0 tx=0).stop 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 shutdown_connections 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd35406be30 0x7fd35406e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd374154f70 0x7fd3741e1ae0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374154b90 0x7fd3741e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 --2- 172.21.15.49:0/2926519483 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3740c6640 0x7fd374159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 >> 172.21.15.49:0/2926519483 conn(0x7fd374074840 msgr2=0x7fd3741538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:51.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 shutdown_connections 2024-05-09T08:56:51.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.313+0000 7fd37a73b640 1 -- 172.21.15.49:0/2926519483 wait complete. 2024-05-09T08:56:51.384 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:51.385+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:51.400 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:51.400 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:51.400 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:51.400 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:51.409 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:56:51.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.411+0000 7fb05c748640 1 Processor -- start 2024-05-09T08:56:51.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.411+0000 7fb05c748640 1 -- start start 2024-05-09T08:56:51.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb054154b90 0x7fb0540cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb054154f70 0x7fb0540cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 0x7fb0540c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb0540c6fd0 con 0x7fb054154b90 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb054072600 con 0x7fb0540c6640 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.412+0000 7fb05c748640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb0540728e0 con 0x7fb054154f70 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb054154b90 0x7fb0540cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb054154b90 0x7fb0540cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41106/0 (socket says 172.21.15.49:41106) 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 -- 172.21.15.49:0/2700340793 learned_addr learned my addr 172.21.15.49:0/2700340793 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05ace5640 1 --2- 172.21.15.49:0/2700340793 >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 0x7fb0540c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb059ce3640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb054154f70 0x7fb0540cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 -- 172.21.15.49:0/2700340793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb054154f70 msgr2=0x7fb0540cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb054154f70 0x7fb0540cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 -- 172.21.15.49:0/2700340793 >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 msgr2=0x7fb0540c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/2700340793 >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 0x7fb0540c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05a4e4640 1 -- 172.21.15.49:0/2700340793 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0540c5880 con 0x7fb054154b90 2024-05-09T08:56:51.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.413+0000 7fb05ace5640 1 --2- 172.21.15.49:0/2700340793 >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 0x7fb0540c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154b90 0x7fb0540cb280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0540cdad0 tx=0x7fb04402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1560fccb4f658ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb0594e2640 1 -- 172.21.15.49:0/2700340793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044036070 con 0x7fb054154b90 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb0594e2640 1 -- 172.21.15.49:0/2700340793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb044031600 con 0x7fb054154b90 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb0594e2640 1 -- 172.21.15.49:0/2700340793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440319d0 con 0x7fb054154b90 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 -- 172.21.15.49:0/2700340793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154b90 msgr2=0x7fb0540cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154b90 0x7fb0540cb280 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0540cdad0 tx=0x7fb04402a730 comp rx=0 tx=0).stop 2024-05-09T08:56:51.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 -- 172.21.15.49:0/2700340793 shutdown_connections 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 --2- 172.21.15.49:0/2700340793 >> v2:172.21.15.183:3300/0 conn(0x7fb0540c6640 0x7fb0540c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb054154f70 0x7fb0540cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 --2- 172.21.15.49:0/2700340793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154b90 0x7fb0540cb280 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 -- 172.21.15.49:0/2700340793 >> 172.21.15.49:0/2700340793 conn(0x7fb054074840 msgr2=0x7fb054074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 -- 172.21.15.49:0/2700340793 shutdown_connections 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.414+0000 7fb05c748640 1 -- 172.21.15.49:0/2700340793 wait complete. 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 Processor -- start 2024-05-09T08:56:51.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 -- start start 2024-05-09T08:56:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0540c6640 0x7fb0540b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb054154b90 0x7fb0541f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0540c6120 con 0x7fb054154f70 2024-05-09T08:56:51.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0540c5e40 con 0x7fb054154b90 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05c748640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb05415af40 con 0x7fb0540c6640 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05ace5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05ace5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41108/0 (socket says 172.21.15.49:41108) 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.416+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 learned_addr learned my addr 172.21.15.49:0/853677571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0540c6640 msgr2=0x7fb0540b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0540c6640 0x7fb0540b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb054154b90 msgr2=0x7fb0541f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb054154b90 0x7fb0541f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0541f40d0 con 0x7fb054154f70 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7f0 tx=0x7fb04c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c015020 con 0x7fb054154f70 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb04c004230 con 0x7fb054154f70 2024-05-09T08:56:51.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05414ddf0 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c007620 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.417+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541f4650 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.418+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb05404f660 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.419+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb04c018020 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.419+0000 7fb052ffd640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 0x7fb04006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.419+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5857+0+0 (secure 0 0 0) 0x7fb04c08b1c0 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.419+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 0x7fb04006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.419+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7fb04c01f020 con 0x7fb054154f70 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.420+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fb0541f1000 0x7fb054151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.420+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb054151070 con 0x7fb0541f1000 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.420+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fb0541f1000 0x7fb054151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.420+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 0x7fb04006e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb044033000 tx=0x7fb04402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.420+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fb0541f1000 0x7fb054151870 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.421+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb054151070 con 0x7fb0541f1000 2024-05-09T08:56:51.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.436+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb05414fce0 con 0x7fb0541f1000 2024-05-09T08:56:51.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.437+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb05414fce0 con 0x7fb0541f1000 2024-05-09T08:56:51.436 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:51.437 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:51.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.437+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0540ce390 0x7fb05414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.437+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb05414fa00 con 0x7fb0540ce390 2024-05-09T08:56:51.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.437+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0540ce390 0x7fb05414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.438+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0540ce390 0x7fb05414c6b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.438+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb05414fa00 con 0x7fb0540ce390 2024-05-09T08:56:51.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.454+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb0540008d0 con 0x7fb0540ce390 2024-05-09T08:56:51.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.455+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0540008d0 con 0x7fb0540ce390 2024-05-09T08:56:51.455 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.456+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fb054061c70 0x7fb054062050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.456+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb0540b5dd0 con 0x7fb054061c70 2024-05-09T08:56:51.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.456+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fb054061c70 0x7fb054062050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.456+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fb054061c70 0x7fb054062050 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.457+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0540b5dd0 con 0x7fb054061c70 2024-05-09T08:56:51.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.471+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb05414da20 con 0x7fb054061c70 2024-05-09T08:56:51.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.473+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb05414da20 con 0x7fb054061c70 2024-05-09T08:56:51.472 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:51.472 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:51.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.473+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fb0541523b0 0x7fb0541527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.473+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb054062aa0 con 0x7fb0541523b0 2024-05-09T08:56:51.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.473+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fb0541523b0 0x7fb0541527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.473+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fb0541523b0 0x7fb0541527f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.474+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb054062aa0 con 0x7fb0541523b0 2024-05-09T08:56:51.485 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:51.486+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:51.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.488+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb054153000 con 0x7fb0541523b0 2024-05-09T08:56:51.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.490+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb054153000 con 0x7fb0541523b0 2024-05-09T08:56:51.489 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:51.490 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:51.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.490+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fb05402c550 0x7fb05402c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.490+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb054062aa0 con 0x7fb05402c550 2024-05-09T08:56:51.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.490+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fb05402c550 0x7fb05402c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.491+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fb05402c550 0x7fb05402c990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.492+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb054062aa0 con 0x7fb05402c550 2024-05-09T08:56:51.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.506+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb054062aa0 con 0x7fb05402c550 2024-05-09T08:56:51.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.509+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb054062aa0 con 0x7fb05402c550 2024-05-09T08:56:51.508 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.509+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fb0542053b0 0x7fb0542077a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.509+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb054062aa0 con 0x7fb0542053b0 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.509+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fb0542053b0 0x7fb0542077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.510+0000 7fb059ce3640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fb0542053b0 0x7fb0542077a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.510+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb054062aa0 con 0x7fb0542053b0 2024-05-09T08:56:51.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.525+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb054062aa0 con 0x7fb0542053b0 2024-05-09T08:56:51.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.527+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb054062aa0 con 0x7fb0542053b0 2024-05-09T08:56:51.526 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.527+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fb0542081f0 0x7fb05420a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.527+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb05402d3e0 con 0x7fb0542081f0 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.527+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fb0542081f0 0x7fb05420a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.527+0000 7fb05ace5640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fb0542081f0 0x7fb05420a5e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.528+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb05402d3e0 con 0x7fb0542081f0 2024-05-09T08:56:51.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.542+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb05402d3e0 con 0x7fb0542081f0 2024-05-09T08:56:51.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.544+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb05402d3e0 con 0x7fb0542081f0 2024-05-09T08:56:51.543 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.544+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fb05420b030 0x7fb05420d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.544+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb0542101f0 con 0x7fb05420b030 2024-05-09T08:56:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.544+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fb05420b030 0x7fb05420d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:51.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.545+0000 7fb05a4e4640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fb05420b030 0x7fb05420d420 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:51.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.546+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0542101f0 con 0x7fb05420b030 2024-05-09T08:56:51.553 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:51.554 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:51.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.560+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb05420d960 con 0x7fb05420b030 2024-05-09T08:56:51.561 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb052ffd640 1 -- 172.21.15.49:0/853677571 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb05420d960 con 0x7fb05420b030 2024-05-09T08:56:51.561 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:51.563 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:51.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fb0541f1000 msgr2=0x7fb054151870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fb0541f1000 0x7fb054151870 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0540ce390 msgr2=0x7fb05414c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0540ce390 0x7fb05414c6b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fb054061c70 msgr2=0x7fb054062050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fb054061c70 0x7fb054062050 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fb0541523b0 msgr2=0x7fb0541527f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fb0541523b0 0x7fb0541527f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fb05402c550 msgr2=0x7fb05402c990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fb05402c550 0x7fb05402c990 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fb0542053b0 msgr2=0x7fb0542077a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 reap_dead start 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fb0542053b0 0x7fb0542077a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 reap_dead start 2024-05-09T08:56:51.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fb0542081f0 msgr2=0x7fb05420a5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fb0542081f0 0x7fb05420a5e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fb05420b030 msgr2=0x7fb05420d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fb05420b030 0x7fb05420d420 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 msgr2=0x7fb04006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05ace5640 1 -- 172.21.15.49:0/853677571 reap_dead start 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 0x7fb04006e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb044033000 tx=0x7fb04402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 msgr2=0x7fb05414b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00c7f0 tx=0x7fb04c00ccb0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 shutdown_connections 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb04006be60 0x7fb04006e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 --2- 172.21.15.49:0/853677571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb054154f70 0x7fb05414b1b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 >> 172.21.15.49:0/853677571 conn(0x7fb054074840 msgr2=0x7fb054071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:51.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.562+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 shutdown_connections 2024-05-09T08:56:51.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:51.563+0000 7fb05c748640 1 -- 172.21.15.49:0/853677571 wait complete. 2024-05-09T08:56:51.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:56:51.613 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:51.614+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:51.669 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:51.669 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:51.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:56:51.722 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:51.723+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:51.780 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:51.781 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:51.788 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:51.789+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:51.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:56:51.889 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:51.890+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:51.931 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:51.931 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:51.939 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:56:52.023 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:52.024+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:52.082 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.082 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.091 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:56:52.092 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:52.092+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:52.192 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:52.193+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:52.234 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.234 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.244 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:56:52.292 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:52.293+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:52.348 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.348 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.357 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:56:52.393 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:52.394+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:52.494 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:52.495+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:52.500 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.500 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:56:52.595 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:52.597+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:52.651 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.651 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.660 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:56:52.697 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:52.698+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:52.798 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:52.799+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:52.803 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.803 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:56:52.899 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:52.900+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:52.914 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:52.914 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:52.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:56:52.999 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:53.000+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:53.046 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.046 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.055 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:56:53.100 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:53.102+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:53.166 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.166 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.176 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:56:53.200 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:53.201+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:53.286 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.286 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:56:53.301 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:53.303+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:53.318 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:56:53.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-09T08:56:53.325 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:56:53.402 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:53.403+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:53.407 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.407 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:56:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.448+0000 7ff9aa872640 1 Processor -- start 2024-05-09T08:56:53.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.448+0000 7ff9aa872640 1 -- start start 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9aa872640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9a4154b90 0x7ff9a40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9aa872640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9aa872640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a4072600 con 0x7ff9a4154f70 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9aa872640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a40728e0 con 0x7ff9a4154b90 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9aa872640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a40c5880 con 0x7ff9a40c6d50 2024-05-09T08:56:53.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff9a4154b90 0x7ff9a40cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9a37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff9a4154f70 0x7ff9a40c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51148/0 (socket says 172.21.15.49:51148) 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9a37fe640 1 -- 172.21.15.49:0/1071500818 learned_addr learned my addr 172.21.15.49:0/1071500818 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.449+0000 7ff9a8e0f640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a3fff640 1 -- 172.21.15.49:0/1071500818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 msgr2=0x7ff9a40cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a3fff640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a3fff640 1 -- 172.21.15.49:0/1071500818 >> v1:172.21.15.49:6789/0 conn(0x7ff9a4154f70 legacy=0x7ff9a40c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a3fff640 1 -- 172.21.15.49:0/1071500818 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a40c5b60 con 0x7ff9a4154b90 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a2ffd640 1 -- 172.21.15.49:0/1071500818 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288983756 0 0) 0x7ff9a4072600 con 0x7ff9a4154f70 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.450+0000 7ff9a3fff640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154b90 0x7ff9a40cb280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9940097a0 tx=0x7ff99402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99699cfb641b847 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9a2ffd640 1 -- 172.21.15.49:0/1071500818 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994037070 con 0x7ff9a4154b90 2024-05-09T08:56:53.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9a2ffd640 1 -- 172.21.15.49:0/1071500818 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff994032440 con 0x7ff9a4154b90 2024-05-09T08:56:53.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9a2ffd640 1 -- 172.21.15.49:0/1071500818 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994032810 con 0x7ff9a4154b90 2024-05-09T08:56:53.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 -- 172.21.15.49:0/1071500818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154b90 msgr2=0x7ff9a40cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154b90 0x7ff9a40cb280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9940097a0 tx=0x7ff99402a210 comp rx=0 tx=0).stop 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 -- 172.21.15.49:0/1071500818 shutdown_connections 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 --2- 172.21.15.49:0/1071500818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154b90 0x7ff9a40cb280 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 -- 172.21.15.49:0/1071500818 >> 172.21.15.49:0/1071500818 conn(0x7ff9a4074840 msgr2=0x7ff9a4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 -- 172.21.15.49:0/1071500818 shutdown_connections 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.451+0000 7ff9aa872640 1 -- 172.21.15.49:0/1071500818 wait complete. 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.452+0000 7ff9aa872640 1 Processor -- start 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 -- start start 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a4154b90 0x7ff9a4159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a415af40 con 0x7ff9a4154f70 2024-05-09T08:56:53.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a40c6120 con 0x7ff9a4154b90 2024-05-09T08:56:53.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9aa872640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a4071820 con 0x7ff9a40c6d50 2024-05-09T08:56:53.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9a8e0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9a8e0f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55540/0 (socket says 172.21.15.49:55540) 2024-05-09T08:56:53.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.453+0000 7ff9a8e0f640 1 -- 172.21.15.49:0/132126803 learned_addr learned my addr 172.21.15.49:0/132126803 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:53.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 -- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 msgr2=0x7ff9a40ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 -- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a4154b90 msgr2=0x7ff9a4159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a4154b90 0x7ff9a4159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 -- 172.21.15.49:0/132126803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a414a8f0 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a8e0f640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007e40 tx=0x7ff99c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c014620 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c019070 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a40cfd70 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c014a00 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.454+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a40d0330 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.455+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a4069060 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.456+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff99c014ba0 con 0x7ff9a4154f70 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.457+0000 7ff9a0ff9640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 0x7ff99006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.457+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5857+0+0 (secure 0 0 0) 0x7ff99c08b7b0 con 0x7ff9a4154f70 2024-05-09T08:56:53.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.457+0000 7ff9a3fff640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 0x7ff99006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.458+0000 7ff9a3fff640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 0x7ff99006e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff994034000 tx=0x7ff99402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.460+0000 7f8764858640 1 Processor -- start 2024-05-09T08:56:53.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.460+0000 7f8764858640 1 -- start start 2024-05-09T08:56:53.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f8764858640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87600707d0 0x7f8760072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9a4069060 con 0x7ff9a4154f70 2024-05-09T08:56:53.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f8764858640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876006fd40 con 0x7f8760072660 2024-05-09T08:56:53.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f8764858640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876015b940 con 0x7f8760154d50 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f8764858640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8760070c40 con 0x7f87600707d0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f875dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87600707d0 0x7f8760072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f875dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87600707d0 0x7f8760072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55546/0 (socket says 172.21.15.49:55546) 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f875ed9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8760072660 0x7f8760072a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51164/0 (socket says 172.21.15.49:51164) 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.461+0000 7f875dd9b640 1 -- 172.21.15.49:0/686828123 learned_addr learned my addr 172.21.15.49:0/686828123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.462+0000 7f875dd9b640 1 -- 172.21.15.49:0/686828123 >> v1:172.21.15.183:6789/0 conn(0x7f8760154d50 legacy=0x7f876006f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.462+0000 7f875dd9b640 1 -- 172.21.15.49:0/686828123 >> v1:172.21.15.49:6789/0 conn(0x7f8760072660 legacy=0x7f8760072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.462+0000 7f875dd9b640 1 -- 172.21.15.49:0/686828123 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876015bc20 con 0x7f87600707d0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.463+0000 7f875dd9b640 1 --2- 172.21.15.49:0/686828123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600707d0 0x7f8760072120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0ba74ff200b59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.463+0000 7f875d59a640 1 -- 172.21.15.49:0/686828123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f87600707d0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.463+0000 7f875d59a640 1 -- 172.21.15.49:0/686828123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754032640 con 0x7f87600707d0 2024-05-09T08:56:53.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.463+0000 7f875d59a640 1 -- 172.21.15.49:0/686828123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f87600707d0 2024-05-09T08:56:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 -- 172.21.15.49:0/686828123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600707d0 msgr2=0x7f8760072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 --2- 172.21.15.49:0/686828123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600707d0 0x7f8760072120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8754009870 tx=0x7f875402a3c0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 -- 172.21.15.49:0/686828123 shutdown_connections 2024-05-09T08:56:53.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 --2- 172.21.15.49:0/686828123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600707d0 0x7f8760072120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 -- 172.21.15.49:0/686828123 >> 172.21.15.49:0/686828123 conn(0x7f87600ce390 msgr2=0x7f87600ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:53.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 -- 172.21.15.49:0/686828123 shutdown_connections 2024-05-09T08:56:53.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.464+0000 7f8764858640 1 -- 172.21.15.49:0/686828123 wait complete. 2024-05-09T08:56:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.465+0000 7f8764858640 1 Processor -- start 2024-05-09T08:56:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.465+0000 7f8764858640 1 -- start start 2024-05-09T08:56:53.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 0x7f87601f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154d50 0x7f8760151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f876006b410 con 0x7f8760072660 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87600c9110 con 0x7f87600707d0 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f8764858640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8760155130 con 0x7f8760154d50 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f875dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f875dd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55548/0 (socket says 172.21.15.49:55548) 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f875dd9b640 1 -- 172.21.15.49:0/3599584187 learned_addr learned my addr 172.21.15.49:0/3599584187 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f875ed9d640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154d50 0x7f8760151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.466+0000 7f875e59c640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 0x7f87601f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 -- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154d50 msgr2=0x7f8760151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154d50 0x7f8760151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 -- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 msgr2=0x7f87601f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 0x7f87601f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 -- 172.21.15.49:0/3599584187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8760152c20 con 0x7f8760072660 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875e59c640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 0x7f87601f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f875dd9b640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f875402a400 tx=0x7f87540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754041070 con 0x7f8760072660 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754030980 con 0x7f8760072660 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f8760072660 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8760152940 con 0x7f8760072660 2024-05-09T08:56:53.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.467+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8760155a20 con 0x7f8760072660 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.468+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8760154900 con 0x7f8760072660 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.469+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8754044070 con 0x7f8760072660 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.470+0000 7f874effd640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 0x7f874006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.470+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5857+0+0 (secure 0 0 0) 0x7f87540ab800 con 0x7f8760072660 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.470+0000 7f875e59c640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 0x7f874006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.471+0000 7f875e59c640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 0x7f874006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.474+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8760154900 con 0x7f8760072660 2024-05-09T08:56:53.503 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:53.505+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:53.565 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.565 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:56:53.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:56:53.604 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:53.605+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:53.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.673+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f876004f660 con 0x7f8760072660 2024-05-09T08:56:53.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.678+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7ff9a404f660 con 0x7ff9a4154f70 2024-05-09T08:56:53.704 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:53.706+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:53.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.720+0000 7f4c6f1a0640 1 Processor -- start 2024-05-09T08:56:53.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.720+0000 7f4c6f1a0640 1 -- start start 2024-05-09T08:56:53.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6f1a0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4c68154b80 0x7f4c6815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6f1a0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c68154f60 con 0x7f4c68071c20 2024-05-09T08:56:53.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6f1a0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c68072120 con 0x7f4c68071820 2024-05-09T08:56:53.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6f1a0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4c680caff0 con 0x7f4c68154b80 2024-05-09T08:56:53.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6e99f640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4c68071c20 0x7f4c680cba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51178/0 (socket says 172.21.15.49:51178) 2024-05-09T08:56:53.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6e99f640 1 -- 172.21.15.49:0/81581389 learned_addr learned my addr 172.21.15.49:0/81581389 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:53.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/81581389 >> v2:172.21.15.183:3300/0 conn(0x7f4c68154b80 0x7f4c6815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.721+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751231842 0 0) 0x7f4c68154f60 con 0x7f4c68071c20 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.722+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c50003520 con 0x7f4c68071c20 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.722+0000 7f4c6e19e640 1 -- 172.21.15.49:0/81581389 >> v1:172.21.15.183:6789/0 conn(0x7f4c68071820 legacy=0x7f4c680cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.722+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1068747575 0 0) 0x7f4c50003520 con 0x7f4c68071c20 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.722+0000 7f4c6e19e640 1 -- 172.21.15.49:0/81581389 >> v1:172.21.15.49:6789/0 conn(0x7f4c68071c20 legacy=0x7f4c680cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.722+0000 7f4c6e19e640 1 -- 172.21.15.49:0/81581389 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c680723e0 con 0x7f4c68154b80 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.723+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/81581389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68154b80 0x7f4c6815b5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c600097a0 tx=0x7f4c6002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e1747647c65ff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.723+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60037070 con 0x7f4c68154b80 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.723+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6002ac50 con 0x7f4c68154b80 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.723+0000 7f4c6d19c640 1 -- 172.21.15.49:0/81581389 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c580019f0 con 0x7f4c68154b80 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/81581389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68154b80 msgr2=0x7f4c6815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/81581389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68154b80 0x7f4c6815b5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c600097a0 tx=0x7f4c6002a1c0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/81581389 shutdown_connections 2024-05-09T08:56:53.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/81581389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68154b80 0x7f4c6815b5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/81581389 >> 172.21.15.49:0/81581389 conn(0x7f4c68074840 msgr2=0x7f4c68074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/81581389 shutdown_connections 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.724+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/81581389 wait complete. 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.725+0000 7f4c6f1a0640 1 Processor -- start 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.725+0000 7f4c6f1a0640 1 -- start start 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68071820 0x7f4c6815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c68154b80 0x7f4c68155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680cb240 con 0x7f4c68071c20 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c680cb5b0 con 0x7f4c68071820 2024-05-09T08:56:53.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6f1a0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4c680ccf60 con 0x7f4c68154b80 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55554/0 (socket says 172.21.15.49:55554) 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 -- 172.21.15.49:0/2244494303 learned_addr learned my addr 172.21.15.49:0/2244494303 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c68154b80 0x7f4c68155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68071820 0x7f4c6815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c68154b80 msgr2=0x7f4c68155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c68154b80 0x7f4c68155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68071820 msgr2=0x7f4c6815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c68071820 0x7f4c6815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6815c190 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00c480 tx=0x7f4c5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.726+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c0052d0 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.727+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c5c005470 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.727+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6815e800 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.727+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c012870 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.727+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681f0b80 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.727+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4c6804f5f0 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.728+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c5c012a10 con 0x7f4c68071c20 2024-05-09T08:56:53.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c66ffd640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 0x7f4c5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5857+0+0 (secure 0 0 0) 0x7f4c5c08b3b0 con 0x7f4c68071c20 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7f4c5c08b840 con 0x7f4c68071c20 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 0x7f4c5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4c681f0d80 0x7f4c680c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c6815ef20 con 0x7f4c681f0d80 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.729+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4c681f0d80 0x7f4c680c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.730+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 0x7f4c5406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c60034000 tx=0x7f4c6002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.730+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4c681f0d80 0x7f4c680c52d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.730+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c6815ef20 con 0x7f4c681f0d80 2024-05-09T08:56:53.741 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:53.741 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:53.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.746+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c6814e680 con 0x7f4c681f0d80 2024-05-09T08:56:53.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.747+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c6814e680 con 0x7f4c681f0d80 2024-05-09T08:56:53.747 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:53.747 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.747+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4c681f1640 0x7f4c68148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.747+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4c68158440 con 0x7f4c681f1640 2024-05-09T08:56:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.748+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4c681f1640 0x7f4c68148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.748+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4c681f1640 0x7f4c68148220 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.749+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c68158440 con 0x7f4c681f1640 2024-05-09T08:56:53.750 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:56:53.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.764+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c680cfee0 con 0x7f4c681f1640 2024-05-09T08:56:53.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.766+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c680cfee0 con 0x7f4c681f1640 2024-05-09T08:56:53.766 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.766+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4c681f4410 0x7f4c681f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.766+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4c681f4d30 con 0x7f4c681f4410 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.766+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4c681f4410 0x7f4c681f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.767+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4c681f4410 0x7f4c681f47f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.767+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c681f4d30 con 0x7f4c681f4410 2024-05-09T08:56:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.783+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c68075080 con 0x7f4c681f4410 2024-05-09T08:56:53.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.784+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c68075080 con 0x7f4c681f4410 2024-05-09T08:56:53.784 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:53.784 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:53.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.785+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4c680622f0 0x7f4c68062710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.785+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4c682032d0 con 0x7f4c680622f0 2024-05-09T08:56:53.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.785+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4c680622f0 0x7f4c68062710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.785+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4c680622f0 0x7f4c68062710 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.786+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c682032d0 con 0x7f4c680622f0 2024-05-09T08:56:53.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.800+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c68158a70 con 0x7f4c680622f0 2024-05-09T08:56:53.801 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.802+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c68158a70 con 0x7f4c680622f0 2024-05-09T08:56:53.801 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:53.802 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:53.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.802+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4c68159190 0x7f4c6802c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.802+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4c681589f0 con 0x7f4c68159190 2024-05-09T08:56:53.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.803+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4c68159190 0x7f4c6802c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.803+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4c68159190 0x7f4c6802c040 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.804+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c681589f0 con 0x7f4c68159190 2024-05-09T08:56:53.806 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:53.807+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:53.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.819+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c681587a0 con 0x7f4c68159190 2024-05-09T08:56:53.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.821+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c681587a0 con 0x7f4c68159190 2024-05-09T08:56:53.820 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.821+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4c6802ca90 0x7f4c6802ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.821+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4c6802d3f0 con 0x7f4c6802ca90 2024-05-09T08:56:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.822+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4c6802ca90 0x7f4c6802ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.822+0000 7f4c6d99d640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4c6802ca90 0x7f4c6802ceb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.823+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c6802d3f0 con 0x7f4c6802ca90 2024-05-09T08:56:53.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.838+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c6802d3f0 con 0x7f4c6802ca90 2024-05-09T08:56:53.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.840+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c6802d3f0 con 0x7f4c6802ca90 2024-05-09T08:56:53.840 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:53.840 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:53.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.840+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4c682079b0 0x7f4c68209da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.840+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4c6820cbf0 con 0x7f4c682079b0 2024-05-09T08:56:53.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.841+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4c682079b0 0x7f4c68209da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.841+0000 7f4c6e99f640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4c682079b0 0x7f4c68209da0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.842+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c6820cbf0 con 0x7f4c682079b0 2024-05-09T08:56:53.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.857+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c6820a360 con 0x7f4c682079b0 2024-05-09T08:56:53.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.859+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c6820a360 con 0x7f4c682079b0 2024-05-09T08:56:53.859 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.859+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4c6820aa80 0x7f4c6820aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.859+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4c68210010 con 0x7f4c6820aa80 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.859+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4c6820aa80 0x7f4c6820aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.860+0000 7f4c6e19e640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4c6820aa80 0x7f4c6820aec0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:53.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.860+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c68210010 con 0x7f4c6820aa80 2024-05-09T08:56:53.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.875+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c6820b510 con 0x7f4c6820aa80 2024-05-09T08:56:53.875 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:53.875 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:53.876 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c66ffd640 1 -- 172.21.15.49:0/2244494303 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c6820b510 con 0x7f4c6820aa80 2024-05-09T08:56:53.876 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4c681f0d80 msgr2=0x7f4c680c52d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4c681f0d80 0x7f4c680c52d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4c681f1640 msgr2=0x7f4c68148220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4c681f1640 0x7f4c68148220 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4c681f4410 msgr2=0x7f4c681f47f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4c681f4410 0x7f4c681f47f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4c680622f0 msgr2=0x7f4c68062710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4c680622f0 0x7f4c68062710 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6e99f640 1 -- 172.21.15.49:0/2244494303 reap_dead start 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4c68159190 msgr2=0x7f4c6802c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4c68159190 0x7f4c6802c040 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4c6802ca90 msgr2=0x7f4c6802ceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4c6802ca90 0x7f4c6802ceb0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4c682079b0 msgr2=0x7f4c68209da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4c682079b0 0x7f4c68209da0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4c6820aa80 msgr2=0x7f4c6820aec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4c6820aa80 0x7f4c6820aec0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.877+0000 7f4c6e99f640 1 -- 172.21.15.49:0/2244494303 reap_dead start 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 msgr2=0x7f4c5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 0x7f4c5406e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c60034000 tx=0x7f4c6002e000 comp rx=0 tx=0).stop 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 msgr2=0x7f4c6815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00c480 tx=0x7f4c5c00c940 comp rx=0 tx=0).stop 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 shutdown_connections 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4c5406be30 0x7f4c5406e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 --2- 172.21.15.49:0/2244494303 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4c68071c20 0x7f4c6815d8f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 >> 172.21.15.49:0/2244494303 conn(0x7f4c68074840 msgr2=0x7f4c680c5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 shutdown_connections 2024-05-09T08:56:53.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:53.878+0000 7f4c6f1a0640 1 -- 172.21.15.49:0/2244494303 wait complete. 2024-05-09T08:56:53.885 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:56:53.906 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:53.907+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:54.007 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:54.008+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:54.037 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:54.038 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:54.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:56:54.108 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:54.109+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:54.198 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517117419999998, 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005543915, 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680886+0000", 2024-05-09T08:56:54.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-05-09T08:56:54.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680905+0000", 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9360000000000001e-05 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681012+0000", 2024-05-09T08:56:54.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107077 2024-05-09T08:56:54.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681023+0000", 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1236000000000001e-05 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116939 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681272+0000", 2024-05-09T08:56:54.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013183499999999999 2024-05-09T08:56:54.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681397+0000", 2024-05-09T08:56:54.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124456 2024-05-09T08:56:54.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685651+0000", 2024-05-09T08:56:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042543770000000002 2024-05-09T08:56:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686328+0000", 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067728299999999999 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686380+0000", 2024-05-09T08:56:54.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2012000000000003e-05 2024-05-09T08:56:54.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686425+0000", 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4305999999999999e-05 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.212 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.212 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:54.212 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517088272999999, 2024-05-09T08:56:54.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060063249999999999, 2024-05-09T08:56:54.212 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:54.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:56:54.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680914+0000", 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7799999999999998e-06 2024-05-09T08:56:54.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681059+0000", 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000144986 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681066+0000", 2024-05-09T08:56:54.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4910000000000002e-06 2024-05-09T08:56:54.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682105+0000", 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010389920000000001 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682185+0000", 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0357000000000003e-05 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682323+0000", 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828000000000001 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685656+0000", 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033325149999999999 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686805+0000", 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001149529 2024-05-09T08:56:54.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686877+0000", 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1303000000000005e-05 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686916+0000", 2024-05-09T08:56:54.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9461e-05 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.223 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517082496999997, 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006182725, 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.96e-07 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.61e-06 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00020153400000000001 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681129+0000", 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2620000000000004e-06 2024-05-09T08:56:54.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682565+0000", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014361650000000001 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682695+0000", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012924400000000001 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682808+0000", 2024-05-09T08:56:54.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113004 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685660+0000", 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002852121 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686999+0000", 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001338802 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687050+0000", 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1625000000000001e-05 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687098+0000", 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8062000000000002e-05 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.229 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517078628999997, 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063447829999999997, 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680920+0000", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9200000000000002e-07 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680945+0000", 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4856000000000001e-05 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681153+0000", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000208385 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681161+0000", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3189999999999995e-06 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683100+0000", 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019385590000000001 2024-05-09T08:56:54.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683175+0000", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4972e-05 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683297+0000", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012193699999999999 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685664+0000", 2024-05-09T08:56:54.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023667929999999998 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687171+0000", 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001507216 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687223+0000", 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2571999999999997e-05 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687264+0000", 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0881999999999999e-05 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517051463999998, 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064597120000000003, 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.235 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:56:54.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1600000000000001e-07 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680950+0000", 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226572 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681193+0000", 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6884e-05 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683542+0000", 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023484650000000001 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683657+0000", 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115277 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683766+0000", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010891399999999999 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685668+0000", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001901743 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687313+0000", 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0016458709999999999 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687362+0000", 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8625999999999998e-05 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687406+0000", 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4276000000000001e-05 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517047001999998, 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066291869999999999, 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4799999999999999e-07 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680954+0000", 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6730000000000001e-06 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681306+0000", 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000351491 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681313+0000", 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9619999999999996e-06 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684111+0000", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027984569999999999 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684202+0000", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1195999999999999e-05 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684353+0000", 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015071299999999999 2024-05-09T08:56:54.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686447+0000", 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020943440000000002 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687488+0000", 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001040336 2024-05-09T08:56:54.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687534+0000", 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6774999999999999e-05 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687580+0000", 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5892e-05 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517042906999997, 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006791755, 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680956+0000", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.34e-07 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680958+0000", 2024-05-09T08:56:54.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-06 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681332+0000", 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00037391099999999999 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681338+0000", 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2639999999999999e-06 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684606+0000", 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032684599999999999 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684723+0000", 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011661499999999999 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684853+0000", 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013001199999999999 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686475+0000", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001621883 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687653+0000", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001178438 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687705+0000", 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1805000000000003e-05 2024-05-09T08:56:54.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687747+0000", 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2143e-05 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517038552999999, 2024-05-09T08:56:54.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069219210000000002, 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6399999999999998e-07 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680968+0000", 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3289999999999994e-06 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681351+0000", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00038298100000000001 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681356+0000", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.101e-06 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685141+0000", 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003785086 2024-05-09T08:56:54.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685240+0000", 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8267e-05 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685406+0000", 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000166644 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686653+0000", 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012463190000000001 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687796+0000", 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011430450000000001 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687849+0000", 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2923999999999997e-05 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687881+0000", 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2861000000000003e-05 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.517014082000003, 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048537759999999998, 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680987+0000", 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4370000000000001e-06 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681414+0000", 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00042719999999999998 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681421+0000", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.601e-06 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682060+0000", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00063917100000000001 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682139+0000", 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8872e-05 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682269+0000", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013019400000000001 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684249+0000", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001979794 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685731+0000", 2024-05-09T08:56:54.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014824930000000001 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685788+0000", 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6986e-05 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685838+0000", 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9755000000000001e-05 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:20 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.474041251999999, 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012308524, 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723958+0000", 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.296e-06 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.395e-06 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723990+0000", 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6914999999999999e-05 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723999+0000", 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5129999999999996e-06 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724055+0000", 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6159000000000002e-05 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724166+0000", 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110532 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.736265+0000", 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012099713999999999 2024-05-09T08:56:54.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:15 2.3 e20/17 2:c27f454c:::mds1_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.472015825, 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061558979999999999, 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725983+0000", 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8000000000000004e-07 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725991+0000", 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0860000000000002e-06 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726006+0000", 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4938e-05 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2429999999999999e-06 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727846+0000", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018329609999999999 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727871+0000", 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4406999999999999e-05 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732095+0000", 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042245640000000001 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732138+0000", 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818999999999998e-05 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467935087999997, 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057959939999999996, 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:56:54.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1199999999999998e-07 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730066+0000", 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.694e-06 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730250+0000", 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018380499999999999 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730260+0000", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7100000000000002e-06 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731230+0000", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097010499999999997 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731303+0000", 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3579999999999997e-05 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731420+0000", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116568 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732988+0000", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015678440000000001 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735786+0000", 2024-05-09T08:56:54.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027978769999999998 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1066000000000003e-05 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735859+0000", 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2332999999999997e-05 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467930690000003, 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048474440000000002, 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730068+0000", 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730070+0000", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.553e-06 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730295+0000", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00022485300000000001 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730301+0000", 2024-05-09T08:56:54.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9580000000000004e-06 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730352+0000", 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1400999999999998e-05 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730483+0000", 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013079300000000001 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010618799999999999 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733630+0000", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030401299999999998 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734823+0000", 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011937950000000001 2024-05-09T08:56:54.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734873+0000", 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9707999999999999e-05 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734915+0000", 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1746999999999999e-05 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467926667999997, 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050848680000000002, 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730072+0000", 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6100000000000002e-07 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730075+0000", 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8490000000000002e-06 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730966+0000", 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089175099999999998 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730979+0000", 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2847000000000001e-05 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731802+0000", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082315399999999999 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731879+0000", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7046999999999996e-05 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732012+0000", 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000133005 2024-05-09T08:56:54.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733927+0000", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001914612 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735049+0000", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011219789999999999 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735113+0000", 2024-05-09T08:56:54.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4362999999999993e-05 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735156+0000", 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2899999999999999e-05 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467919227000003, 2024-05-09T08:56:54.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.004901702, 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730081+0000", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8220000000000001e-06 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731051+0000", 2024-05-09T08:56:54.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096957499999999997 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731057+0000", 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9560000000000002e-06 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731629+0000", 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057226399999999995 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731714+0000", 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5661999999999994e-05 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732235+0000", 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052096200000000003 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733613+0000", 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013775339999999999 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734929+0000", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001316198 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734954+0000", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4477999999999998e-05 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734981+0000", 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6933000000000001e-05 2024-05-09T08:56:54.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467915990999998, 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053901399999999999, 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8700000000000002e-07 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730085+0000", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7609999999999999e-06 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731070+0000", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098512399999999998 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731083+0000", 2024-05-09T08:56:54.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3033999999999999e-05 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732979+0000", 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001895276 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733089+0000", 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110542 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733211+0000", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121471 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734283+0000", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001071904 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735386+0000", 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011033620000000001 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735437+0000", 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0735999999999997e-05 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735472+0000", 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5642999999999997e-05 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.304 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467911882999999, 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054919699999999997, 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730087+0000", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7599999999999997e-07 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730089+0000", 2024-05-09T08:56:54.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731097+0000", 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001008412 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731102+0000", 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5589999999999999e-06 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732441+0000", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00133946 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732543+0000", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010173199999999999 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732613+0000", 2024-05-09T08:56:54.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9788000000000003e-05 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734007+0000", 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013943499999999999 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735476+0000", 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014689709999999999 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735536+0000", 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0408999999999997e-05 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735578+0000", 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1897000000000002e-05 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.310 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467908291000001, 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059167780000000001, 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5699999999999999e-07 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730096+0000", 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0599999999999996e-06 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731114+0000", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001017525 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731118+0000", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7639999999999996e-06 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731633+0000", 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00051442600000000001 2024-05-09T08:56:54.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731741+0000", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010863599999999999 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731906+0000", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016487199999999999 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733654+0000", 2024-05-09T08:56:54.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001747738 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735909+0000", 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022545 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735950+0000", 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1201000000000001e-05 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.736007+0000", 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6799000000000003e-05 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467894428999998, 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055945150000000004, 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.316 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:56:54.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000002e-07 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730113+0000", 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1020000000000003e-06 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731131+0000", 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010174279999999999 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731135+0000", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4449999999999998e-06 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733092+0000", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019572190000000001 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733229+0000", 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013633499999999999 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733345+0000", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116647 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734293+0000", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094760700000000005 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735618+0000", 2024-05-09T08:56:54.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013248260000000001 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735667+0000", 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9468000000000001e-05 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735698+0000", 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1053000000000002e-05 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 36.467883604999997, 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057439600000000002, 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:54.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730115+0000", 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730117+0000", 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7940000000000001e-06 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731147+0000", 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010300610000000001 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731151+0000", 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7269999999999998e-06 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733642+0000", 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024900780000000002 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733758+0000", 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116214 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735858+0000", 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021007040000000001 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.326 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:54.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:56:54.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.317+0000 7f874effd640 1 -- 172.21.15.49:0/3599584187 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v27) ==== 87+0+0 (secure 0 0 0) 0x7f8754079ad0 con 0x7f8760072660 2024-05-09T08:56:54.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.317+0000 7ff9a0ff9640 1 -- 172.21.15.49:0/132126803 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v27) ==== 113+0+0 (secure 0 0 0) 0x7ff99c05a0c0 con 0x7ff9a4154f70 2024-05-09T08:56:54.329 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-05-09T08:56:54.329 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:56:54.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 msgr2=0x7f874006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 0x7f874006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8750007fd0 tx=0x7f8750009040 comp rx=0 tx=0).stop 2024-05-09T08:56:54.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 msgr2=0x7f876014cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 msgr2=0x7ff99006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 0x7ff99006e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff994034000 tx=0x7ff99402c000 comp rx=0 tx=0).stop 2024-05-09T08:56:54.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 msgr2=0x7ff9a4149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007e40 tx=0x7ff99c00e4b0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.319+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f875402a400 tx=0x7f87540046c0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 shutdown_connections 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99006be60 0x7ff99006e320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a4154f70 0x7ff9a4149ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a4154b90 0x7ff9a4159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 shutdown_connections 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874006be30 0x7f874006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154d50 0x7f8760151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760072660 0x7f876014cdb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 --2- 172.21.15.49:0/3599584187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600707d0 0x7f87601f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 --2- 172.21.15.49:0/132126803 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a40c6d50 0x7ff9a40ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 >> 172.21.15.49:0/132126803 conn(0x7ff9a4074840 msgr2=0x7ff9a40b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:54.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 >> 172.21.15.49:0/3599584187 conn(0x7f87600ce390 msgr2=0x7f87601599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:54.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 shutdown_connections 2024-05-09T08:56:54.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7f8764858640 1 -- 172.21.15.49:0/3599584187 wait complete. 2024-05-09T08:56:54.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 shutdown_connections 2024-05-09T08:56:54.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.320+0000 7ff9aa872640 1 -- 172.21.15.49:0/132126803 wait complete. 2024-05-09T08:56:54.333 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:54.328+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:54.335 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-09T08:56:54.428 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:54.429+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:54.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.483+0000 7f15ebb48640 1 Processor -- start 2024-05-09T08:56:54.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.483+0000 7f15ebb48640 1 -- start start 2024-05-09T08:56:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.484+0000 7f15ebb48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.484+0000 7f15ebb48640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f15e40c6d50 0x7f15e40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ebb48640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e4072600 con 0x7f15e4154f70 2024-05-09T08:56:54.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ebb48640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f15e40728e0 con 0x7f15e40c6d50 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ebb48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f15e40c5880 con 0x7f15e4154b90 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ea0e5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f15e40c6d50 0x7f15e40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15e90e3640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f15e4154f70 0x7f15e40c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51186/0 (socket says 172.21.15.49:51186) 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15e90e3640 1 -- 172.21.15.49:0/2420939776 learned_addr learned my addr 172.21.15.49:0/2420939776 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15e98e4640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e40cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ea0e5640 1 -- 172.21.15.49:0/2420939776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 msgr2=0x7f15e40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ea0e5640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e40cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15e88e2640 1 -- 172.21.15.49:0/2420939776 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159042635 0 0) 0x7f15e4072600 con 0x7f15e4154f70 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ea0e5640 1 -- 172.21.15.49:0/2420939776 >> v1:172.21.15.49:6789/0 conn(0x7f15e4154f70 legacy=0x7f15e40c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.485+0000 7f15ea0e5640 1 -- 172.21.15.49:0/2420939776 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e40c5b60 con 0x7f15e40c6d50 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15ea0e5640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40cb7c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009d90 tx=0x7f15dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e351d186962ff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15e88e2640 1 -- 172.21.15.49:0/2420939776 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc007740 con 0x7f15e40c6d50 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15e88e2640 1 -- 172.21.15.49:0/2420939776 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc0078e0 con 0x7f15e40c6d50 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15e88e2640 1 -- 172.21.15.49:0/2420939776 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc007cb0 con 0x7f15e40c6d50 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15ebb48640 1 -- 172.21.15.49:0/2420939776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 msgr2=0x7f15e40cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15ebb48640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40cb7c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009d90 tx=0x7f15dc02a6f0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15ebb48640 1 -- 172.21.15.49:0/2420939776 shutdown_connections 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.486+0000 7f15ebb48640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40cb7c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.487+0000 7f15ebb48640 1 --2- 172.21.15.49:0/2420939776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e40cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.487+0000 7f15ebb48640 1 -- 172.21.15.49:0/2420939776 >> 172.21.15.49:0/2420939776 conn(0x7f15e4074840 msgr2=0x7f15e4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.487+0000 7f15ebb48640 1 -- 172.21.15.49:0/2420939776 shutdown_connections 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.487+0000 7f15ebb48640 1 -- 172.21.15.49:0/2420939776 wait complete. 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 Processor -- start 2024-05-09T08:56:54.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 -- start start 2024-05-09T08:56:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15e4154f70 0x7f15e414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f15e415af40 con 0x7f15e40c6d50 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40c6120 con 0x7f15e4154f70 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.488+0000 7f15ebb48640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f15e4071820 con 0x7f15e4154b90 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55578/0 (socket says 172.21.15.49:55578) 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 -- 172.21.15.49:0/4135236690 learned_addr learned my addr 172.21.15.49:0/4135236690 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e90e3640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15ea0e5640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15e4154f70 0x7f15e414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 -- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 msgr2=0x7f15e4151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e4151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 -- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15e4154f70 msgr2=0x7f15e414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15e4154f70 0x7f15e414f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 -- 172.21.15.49:0/4135236690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41f4620 con 0x7f15e40c6d50 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15e98e4640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f15e000d830 tx=0x7f15e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:56:54.491 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:54.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000b6c0 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e0019070 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41f48a0 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000ba80 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.489+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41512c0 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.490+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e404f660 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.491+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15e001e460 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.492+0000 7f15da7fc640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 0x7f15c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.492+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5857+0+0 (secure 0 0 0) 0x7f15e008b560 con 0x7f15e40c6d50 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.492+0000 7f15e90e3640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 0x7f15c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:54.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.493+0000 7f15e90e3640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 0x7f15c806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15e40cada0 tx=0x7f15d4007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:54.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.497+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15e41f4620 con 0x7f15e40c6d50 2024-05-09T08:56:54.499 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:56:54.540 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:54.541+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:54.631 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:54.632+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:54.649 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:54.649 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:56:54.649 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:56:54.650 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:56:54.650 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:54.659 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:56:54.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:54.693+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f15e40b54f0 con 0x7f15e40c6d50 2024-05-09T08:56:54.736 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:54.737+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:54.805 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.123360529999999, 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:54.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:56:54.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.123340558999999, 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:56:54.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:54.814 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.122511793000001, 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:54.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:56:54.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.121508900999999, 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:56:54.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.121206069999999, 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.825 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:56:54.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.829 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.080019512, 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:56:54.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:56:54.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:56:54.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:56:54.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.836 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.080007486, 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:56:54.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.079994016000001, 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:56:54.841 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.078969614000002, 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:56:54.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.078542325000001, 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:56:54.850 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:56:54.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:56:54.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.078425985000003, 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:56:54.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.077151989000001, 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.075402588999999, 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:56:54.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:56:54.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:54.867 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.075074444999998, 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:56:54.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.064945817000002, 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:56:54.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:56:54.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.074202816000003, 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:56:54.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:54.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.074193430000001, 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:56:54.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.074182385999997, 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:56:54.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.072447390000001, 2024-05-09T08:56:54.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:54.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:56:54.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 25.042543183999999, 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:54.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:54.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:54.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:54.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:54.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:54.904 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:54.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:56:54.906 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:54.906+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:55.007 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:55.008+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:55.021 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.021 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:55.021 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:55.021 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:56:55.114 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:55.115+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:55.173 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.174 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:55.174 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:55.174 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:55.174 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.182 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:56:55.223 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:55.223+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:55.310 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:55.311+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:55.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.318+0000 7f15da7fc640 1 -- 172.21.15.49:0/4135236690 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v28) ==== 113+0+0 (secure 0 0 0) 0x7f15e0059f00 con 0x7f15e40c6d50 2024-05-09T08:56:55.317 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 msgr2=0x7f15c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 0x7f15c806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15e40cada0 tx=0x7f15d4007c00 comp rx=0 tx=0).stop 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 msgr2=0x7f15e40c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f15e000d830 tx=0x7f15e000dcf0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 shutdown_connections 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f15c806be30 0x7f15c806e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15e4154f70 0x7f15e414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15e4154b90 0x7f15e4151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 --2- 172.21.15.49:0/4135236690 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f15e40c6d50 0x7f15e40c80d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.321+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 >> 172.21.15.49:0/4135236690 conn(0x7f15e4074840 msgr2=0x7f15e40b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.322+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 shutdown_connections 2024-05-09T08:56:55.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.322+0000 7f15ebb48640 1 -- 172.21.15.49:0/4135236690 wait complete. 2024-05-09T08:56:55.329 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.329 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:55.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:55.329 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:55.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.649102302999999, 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:56:55.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:56:55.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.645682540999999, 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:56:55.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:55.340 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.645456031999998, 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:55.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.645402738999998, 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:56:55.345 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:56:55.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:55.349 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.597079168, 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:56:55.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.596553292999999, 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:56:55.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.596305960999999, 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.358 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:56:55.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:56:55.362 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.590866757999997, 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:56:55.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.367 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.582907594000002, 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:56:55.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:55.371 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.582343415, 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:56:55.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.582256342000001, 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.376 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:56:55.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:56:55.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.582187284, 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.381 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:56:55.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.580828974999999, 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.386 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:56:55.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.390 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.580783728, 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.580528461, 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:56:55.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:56:55.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 31.580489764999999, 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:56:55.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:56:55.404 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.567546706999998, 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:56:55.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.566907758999999, 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.413 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.566353814999999, 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:56:55.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 25.563499676999999, 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.418 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:56:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.422 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.424 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-09T08:56:55.424 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:55.424+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:55.425 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:56:55.525 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:55.525+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:55.530 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:55.530 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:56:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.582+0000 7f98ba0d2640 1 Processor -- start 2024-05-09T08:56:55.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.582+0000 7f98ba0d2640 1 -- start start 2024-05-09T08:56:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98ba0d2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98b40c73a0 0x7f98b406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98ba0d2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b40c4da0 con 0x7f98b415af60 2024-05-09T08:56:55.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98ba0d2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b40c5060 con 0x7f98b4151be0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98ba0d2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f98b40c5c70 con 0x7f98b40c73a0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f98b40c73a0 0x7f98b406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b2ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f98b415af60 0x7f98b40c6c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51194/0 (socket says 172.21.15.49:51194) 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b2ffd640 1 -- 172.21.15.49:0/1849767011 learned_addr learned my addr 172.21.15.49:0/1849767011 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b3fff640 1 -- 172.21.15.49:0/1849767011 >> v1:172.21.15.183:6789/0 conn(0x7f98b4151be0 legacy=0x7f98b40c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b3fff640 1 -- 172.21.15.49:0/1849767011 >> v1:172.21.15.49:6789/0 conn(0x7f98b415af60 legacy=0x7f98b40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b3fff640 1 -- 172.21.15.49:0/1849767011 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b40c5340 con 0x7f98b40c73a0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.583+0000 7f98b27fc640 1 -- 172.21.15.49:0/1849767011 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1821471361 0 0) 0x7f98b40c4da0 con 0x7f98b415af60 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.584+0000 7f98b3fff640 1 --2- 172.21.15.49:0/1849767011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b40c73a0 0x7f98b406dac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07fb7373faeccb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.584+0000 7f98b27fc640 1 -- 172.21.15.49:0/1849767011 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac037070 con 0x7f98b40c73a0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.584+0000 7f98b27fc640 1 -- 172.21.15.49:0/1849767011 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac007740 con 0x7f98b40c73a0 2024-05-09T08:56:55.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.584+0000 7f98b27fc640 1 -- 172.21.15.49:0/1849767011 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac007b10 con 0x7f98b40c73a0 2024-05-09T08:56:55.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 -- 172.21.15.49:0/1849767011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b40c73a0 msgr2=0x7f98b406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/1849767011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b40c73a0 0x7f98b406dac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac02a780 comp rx=0 tx=0).stop 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 -- 172.21.15.49:0/1849767011 shutdown_connections 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/1849767011 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b40c73a0 0x7f98b406dac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 -- 172.21.15.49:0/1849767011 >> 172.21.15.49:0/1849767011 conn(0x7f98b40cc290 msgr2=0x7f98b40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 -- 172.21.15.49:0/1849767011 shutdown_connections 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.585+0000 7f98ba0d2640 1 -- 172.21.15.49:0/1849767011 wait complete. 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.586+0000 7f98ba0d2640 1 Processor -- start 2024-05-09T08:56:55.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.586+0000 7f98ba0d2640 1 -- start start 2024-05-09T08:56:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4151be0 0x7f98b41ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98b415af60 0x7f98b414be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:55.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4151fc0 con 0x7f98b414d4d0 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98b40c5900 con 0x7f98b4151be0 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98ba0d2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f98b4152140 con 0x7f98b415af60 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55608/0 (socket says 172.21.15.49:55608) 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 -- 172.21.15.49:0/2288321226 learned_addr learned my addr 172.21.15.49:0/2288321226 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b2ffd640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98b415af60 0x7f98b414be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b37fe640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4151be0 0x7f98b41ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 -- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98b415af60 msgr2=0x7f98b414be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98b415af60 0x7f98b414be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 -- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4151be0 msgr2=0x7f98b41ef1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4151be0 0x7f98b41ef1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:55.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.587+0000 7f98b3fff640 1 -- 172.21.15.49:0/2288321226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b41f29f0 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f98b3fff640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac037040 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ac032a80 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b41f2430 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac003990 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.588+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41ee6d0 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.589+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b404f660 con 0x7f98b414d4d0 2024-05-09T08:56:55.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.589+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98ac005d10 con 0x7f98b414d4d0 2024-05-09T08:56:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.590+0000 7f989bfff640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 0x7f98a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.590+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5857+0+0 (secure 0 0 0) 0x7f98ac0ab980 con 0x7f98b414d4d0 2024-05-09T08:56:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.590+0000 7f98b37fe640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 0x7f98a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:55.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.591+0000 7f98b37fe640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 0x7f98a006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000f60 tx=0x7f98a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:55.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.594+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f98ac04b020 con 0x7f98b414d4d0 2024-05-09T08:56:55.626 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:55.626+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:55.682 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.682 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:55.682 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:55.682 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:55.682 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.691 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:56:55.732 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:55.732+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:55.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:55.789+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f98b4067160 con 0x7f98b414d4d0 2024-05-09T08:56:55.828 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:55.829+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:55.834 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:55.834 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.109801410999999, 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:56:55.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:56:55.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.109780811999997, 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:56:55.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:56:55.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.846 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.109775941000002, 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:56:55.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.106217641999997, 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:56:55.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:56:55.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:56:55.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.106209413999999, 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:56:55.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.106205605, 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.863 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:56:55.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:56:55.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.103649576999999, 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:56:55.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:56:55.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 32.089148770000001, 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:56:55.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 32.089133298999997, 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:55.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:56:55.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:56:55.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.886 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 32.089128555000002, 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:56:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 32.089122138, 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:56:55.892 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:56:55.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:56:55.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073145919000002, 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.898 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:56:55.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.903 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073133216999999, 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:56:55.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:56:55.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073125300000001, 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:56:55.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.915 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073122791999999, 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:56:55.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:56:55.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073120328000002, 2024-05-09T08:56:55.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:55.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:56:55.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:56:55.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073105033000001, 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.927 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:55.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:56:55.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:55.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073100711999999, 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:56:55.933 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.937 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.073097572000002, 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:56:55.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:56:55.941 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 26.070637772000001, 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:55.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:56:55.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:55.946 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:55.947 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:55.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:56:55.948 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:55.949+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:55.949 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:56:56.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.049+0000 7f6f9299e640 1 Processor -- start 2024-05-09T08:56:56.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 -- start start 2024-05-09T08:56:56.050 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:56.049+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c154f60 con 0x7f6f8c154b80 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8c0caff0 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.050+0000 7f6f9299e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cb5b0 con 0x7f6f8c0ccd90 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9219d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:51260/0 (socket says 172.21.15.49:51260) 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 -- 172.21.15.49:0/4201119663 learned_addr learned my addr 172.21.15.49:0/4201119663 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9199c640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6f8c154b80 0x7f6f8c071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49574/0 (socket says 172.21.15.49:49574) 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9219d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:34906/0 (socket says 172.21.15.49:34906) 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 -- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 msgr2=0x7f6f8c0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 -- 172.21.15.49:0/4201119663 >> v1:172.21.15.183:6789/0 conn(0x7f6f8c154b80 legacy=0x7f6f8c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9119b640 1 -- 172.21.15.49:0/4201119663 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c072120 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.051+0000 7f6f9219d640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9119b640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55e6b4bbe36b5cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9099a640 1 -- 172.21.15.49:0/4201119663 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80037070 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9099a640 1 -- 172.21.15.49:0/4201119663 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f80032440 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9099a640 1 -- 172.21.15.49:0/4201119663 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80032810 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.053 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:56.053 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 -- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c15b5a0 msgr2=0x7f6f8c0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8002a270 comp rx=0 tx=0).stop 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 -- 172.21.15.49:0/4201119663 shutdown_connections 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c0ccd90 0x7f6f8c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 --2- 172.21.15.49:0/4201119663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c15b5a0 0x7f6f8c0cc850 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 -- 172.21.15.49:0/4201119663 >> 172.21.15.49:0/4201119663 conn(0x7f6f8c074840 msgr2=0x7f6f8c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:56.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 -- 172.21.15.49:0/4201119663 shutdown_connections 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.052+0000 7f6f9299e640 1 -- 172.21.15.49:0/4201119663 wait complete. 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.054+0000 7f6f9299e640 1 Processor -- start 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.054+0000 7f6f9299e640 1 -- start start 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c0ccd90 0x7f6f8c144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 0x7f6f8c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 0x7f6f8c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cb240 con 0x7f6f8c154b80 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0726c0 con 0x7f6f8c0ccd90 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9299e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c15b9a0 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9119b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 0x7f6f8c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9199c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c0ccd90 0x7f6f8c144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9119b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 0x7f6f8c0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55624/0 (socket says 172.21.15.49:55624) 2024-05-09T08:56:56.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 0x7f6f8c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9119b640 1 -- 172.21.15.49:0/492516505 learned_addr learned my addr 172.21.15.49:0/492516505 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c0ccd90 msgr2=0x7f6f8c144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c0ccd90 0x7f6f8c144c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 msgr2=0x7f6f8c0c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 0x7f6f8c0c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0c9c30 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6f8c154b80 0x7f6f8c0c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f8c0ccd90 0x7f6f8c144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.055+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 0x7f6f8c14c580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400efc0 tx=0x7f6f8400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.056+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84018030 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.056+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f8400cd30 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.056+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c14ed70 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.056+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84003bb0 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.056+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c14f330 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.057+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f6f8c04f5f0 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.057+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f84005d10 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f7e7fc640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6f7006be30 0x7f6f7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 5857+0+0 (secure 0 0 0) 0x7f6f8408b6f0 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6f7006be30 0x7f6f7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v28) ==== 54+0+16 (secure 0 0 0) 0x7f6f84022070 con 0x7f6f8c15b5a0 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6f8c14cac0 0x7f6f8c14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c151150 con 0x7f6f8c14cac0 2024-05-09T08:56:56.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6f8c14cac0 0x7f6f8c14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6f7006be30 0x7f6f7006e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f88007fd0 tx=0x7f6f88007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.058+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6f8c14cac0 0x7f6f8c14f570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.059+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c151150 con 0x7f6f8c14cac0 2024-05-09T08:56:56.062 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:56:56.073 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.074+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c150e70 con 0x7f6f8c14cac0 2024-05-09T08:56:56.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.075+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c150e70 con 0x7f6f8c14cac0 2024-05-09T08:56:56.075 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.076+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f6f8c0ce860 0x7f6f8c0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.076+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c0cf180 con 0x7f6f8c0ce860 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.076+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f6f8c0ce860 0x7f6f8c0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.076+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f6f8c0ce860 0x7f6f8c0cec40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.077+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c0cf180 con 0x7f6f8c0ce860 2024-05-09T08:56:56.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.092+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c0cbda0 con 0x7f6f8c0ce860 2024-05-09T08:56:56.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.093+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c0cbda0 con 0x7f6f8c0ce860 2024-05-09T08:56:56.093 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:56.093 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:56.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.093+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6f8c1f4100 0x7f6f8c1f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.093 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.093+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c1f4aa0 con 0x7f6f8c1f4100 2024-05-09T08:56:56.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.094+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6f8c1f4100 0x7f6f8c1f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.094+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6f8c1f4100 0x7f6f8c1f4560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.094+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c1f4aa0 con 0x7f6f8c1f4100 2024-05-09T08:56:56.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.109+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c061f40 con 0x7f6f8c1f4100 2024-05-09T08:56:56.109 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.110+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c061f40 con 0x7f6f8c1f4100 2024-05-09T08:56:56.109 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:56.110 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:56.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.110+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6f8c0c5d00 0x7f6f8c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.110+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c062a20 con 0x7f6f8c0c5d00 2024-05-09T08:56:56.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.110+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6f8c0c5d00 0x7f6f8c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.111+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6f8c0c5d00 0x7f6f8c0c6160 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.111+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c062a20 con 0x7f6f8c0c5d00 2024-05-09T08:56:56.124 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.125+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c0c6970 con 0x7f6f8c0c5d00 2024-05-09T08:56:56.126 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.127+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c0c6970 con 0x7f6f8c0c5d00 2024-05-09T08:56:56.126 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.127+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f6f8c02c040 0x7f6f8c02c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.127+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c062a20 con 0x7f6f8c02c040 2024-05-09T08:56:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.127+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f6f8c02c040 0x7f6f8c02c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.128+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f6f8c02c040 0x7f6f8c02c460 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.129+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c062a20 con 0x7f6f8c02c040 2024-05-09T08:56:56.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.143+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c062a20 con 0x7f6f8c02c040 2024-05-09T08:56:56.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.145+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c062a20 con 0x7f6f8c02c040 2024-05-09T08:56:56.144 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:56.145 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:56.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.145+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f6f8c02ceb0 0x7f6f8c206fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.145+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c02d310 con 0x7f6f8c02ceb0 2024-05-09T08:56:56.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.145+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f6f8c02ceb0 0x7f6f8c206fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.145 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.146+0000 7f6f9119b640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f6f8c02ceb0 0x7f6f8c206fd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.146+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c02d310 con 0x7f6f8c02ceb0 2024-05-09T08:56:56.150 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:56.150+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:56.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.161+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c02d310 con 0x7f6f8c02ceb0 2024-05-09T08:56:56.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.163+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c02d310 con 0x7f6f8c02ceb0 2024-05-09T08:56:56.162 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.163+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6f8c207a20 0x7f6f8c209e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.163+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c20cbe0 con 0x7f6f8c207a20 2024-05-09T08:56:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.163+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6f8c207a20 0x7f6f8c209e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.163 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.163+0000 7f6f9219d640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6f8c207a20 0x7f6f8c209e10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.164+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c20cbe0 con 0x7f6f8c207a20 2024-05-09T08:56:56.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.178+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c20a350 con 0x7f6f8c207a20 2024-05-09T08:56:56.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.180+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c20a350 con 0x7f6f8c207a20 2024-05-09T08:56:56.179 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:56.180 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:56.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.180+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6f8c20aa70 0x7f6f8c20aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.180+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6f8c20ffd0 con 0x7f6f8c20aa70 2024-05-09T08:56:56.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.181+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6f8c20aa70 0x7f6f8c20aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.181+0000 7f6f9199c640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6f8c20aa70 0x7f6f8c20aeb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.182+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8c20ffd0 con 0x7f6f8c20aa70 2024-05-09T08:56:56.195 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.196+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6f8c20b440 con 0x7f6f8c20aa70 2024-05-09T08:56:56.197 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f7e7fc640 1 -- 172.21.15.49:0/492516505 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6f8c20b440 con 0x7f6f8c20aa70 2024-05-09T08:56:56.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6f8c14cac0 msgr2=0x7f6f8c14f570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6f8c14cac0 0x7f6f8c14f570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f6f8c0ce860 msgr2=0x7f6f8c0cec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f6f8c0ce860 0x7f6f8c0cec40 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6f8c1f4100 msgr2=0x7f6f8c1f4560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f6f8c1f4100 0x7f6f8c1f4560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.199 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6f8c0c5d00 msgr2=0x7f6f8c0c6160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6f8c0c5d00 0x7f6f8c0c6160 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 reap_dead start 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f6f8c02c040 msgr2=0x7f6f8c02c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f6f8c02c040 0x7f6f8c02c460 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 reap_dead start 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f6f8c02ceb0 msgr2=0x7f6f8c206fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f6f8c02ceb0 0x7f6f8c206fd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6f8c207a20 msgr2=0x7f6f8c209e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6f8c207a20 0x7f6f8c209e10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6f8c20aa70 msgr2=0x7f6f8c20aeb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6f8c20aa70 0x7f6f8c20aeb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6f7006be30 msgr2=0x7f6f7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6f7006be30 0x7f6f7006e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f88007fd0 tx=0x7f6f88007e80 comp rx=0 tx=0).stop 2024-05-09T08:56:56.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 msgr2=0x7f6f8c14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 0x7f6f8c14c580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400efc0 tx=0x7f6f8400c490 comp rx=0 tx=0).stop 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.198+0000 7f6f9219d640 1 -- 172.21.15.49:0/492516505 reap_dead start 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.199+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 shutdown_connections 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.199+0000 7f6f9299e640 1 --2- 172.21.15.49:0/492516505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f8c15b5a0 0x7f6f8c14c580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.199+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 >> 172.21.15.49:0/492516505 conn(0x7f6f8c074840 msgr2=0x7f6f8c151950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.199+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 shutdown_connections 2024-05-09T08:56:56.201 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.199+0000 7f6f9299e640 1 -- 172.21.15.49:0/492516505 wait complete. 2024-05-09T08:56:56.205 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.205 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.214 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:56:56.256 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:56.256+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:56.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.330+0000 7f989bfff640 1 -- 172.21.15.49:0/2288321226 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v29) ==== 113+0+0 (secure 0 0 0) 0x7f98ac042030 con 0x7f98b414d4d0 2024-05-09T08:56:56.330 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 msgr2=0x7f98a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 0x7f98a006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000f60 tx=0x7f98a4009040 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 msgr2=0x7f98b4071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009d90 tx=0x7f98ac004a40 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 shutdown_connections 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f98a006be30 0x7f98a006e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f98b414d4d0 0x7f98b4071370 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98b415af60 0x7f98b414be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 --2- 172.21.15.49:0/2288321226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4151be0 0x7f98b41ef1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 >> 172.21.15.49:0/2288321226 conn(0x7f98b40cc290 msgr2=0x7f98b406bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:56.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 shutdown_connections 2024-05-09T08:56:56.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.333+0000 7f98ba0d2640 1 -- 172.21.15.49:0/2288321226 wait complete. 2024-05-09T08:56:56.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:56:56.344 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-09T08:56:56.361 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.361 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.370 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:56:56.385 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:56.386+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:56.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.450+0000 7faa00083640 1 Processor -- start 2024-05-09T08:56:56.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.450+0000 7faa00083640 1 -- start start 2024-05-09T08:56:56.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7faa00083640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 0x7fa9f80cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7faa00083640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f8072600 con 0x7fa9f80c6d50 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7faa00083640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9f80728e0 con 0x7fa9f8154f70 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7faa00083640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f80c5880 con 0x7fa9f8154b90 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7fa9fe620640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa9f80c6d50 0x7fa9f8071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51196/0 (socket says 172.21.15.49:51196) 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.451+0000 7fa9fe620640 1 -- 172.21.15.49:0/650988276 learned_addr learned my addr 172.21.15.49:0/650988276 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fde1f640 1 --2- 172.21.15.49:0/650988276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 0x7fa9f80cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3295316977 0 0) 0x7fa9f8072600 con 0x7fa9f80c6d50 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9e4003520 con 0x7fa9f80c6d50 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373184425 0 0) 0x7fa9f80728e0 con 0x7fa9f8154f70 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fde1f640 1 -- 172.21.15.49:0/650988276 >> v1:172.21.15.183:6789/0 conn(0x7fa9f8154f70 legacy=0x7fa9f80c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fde1f640 1 -- 172.21.15.49:0/650988276 >> v1:172.21.15.49:6789/0 conn(0x7fa9f80c6d50 legacy=0x7fa9f8071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.452+0000 7fa9fde1f640 1 -- 172.21.15.49:0/650988276 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f80c5b60 con 0x7fa9f8154b90 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.453+0000 7fa9fde1f640 1 --2- 172.21.15.49:0/650988276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 0x7fa9f80cb280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40097d0 tx=0x7fa9f402a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d125e820dcffd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.453+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4037070 con 0x7fa9f8154b90 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.453+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f4032440 con 0x7fa9f8154b90 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.453+0000 7fa9fce1d640 1 -- 172.21.15.49:0/650988276 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4032810 con 0x7fa9f8154b90 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 -- 172.21.15.49:0/650988276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 msgr2=0x7fa9f80cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 --2- 172.21.15.49:0/650988276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 0x7fa9f80cb280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40097d0 tx=0x7fa9f402a2a0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 -- 172.21.15.49:0/650988276 shutdown_connections 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 --2- 172.21.15.49:0/650988276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154b90 0x7fa9f80cb280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 -- 172.21.15.49:0/650988276 >> 172.21.15.49:0/650988276 conn(0x7fa9f8074840 msgr2=0x7fa9f8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 -- 172.21.15.49:0/650988276 shutdown_connections 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.454+0000 7faa00083640 1 -- 172.21.15.49:0/650988276 wait complete. 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.455+0000 7faa00083640 1 Processor -- start 2024-05-09T08:56:56.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.455+0000 7faa00083640 1 -- start start 2024-05-09T08:56:56.456 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:56.453+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:56.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9f80c6d50 0x7fa9f81598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 0x7fa9f80b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f815af40 con 0x7fa9f8154b90 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f80c6120 con 0x7fa9f80c6d50 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7faa00083640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f80cb7c0 con 0x7fa9f8154f70 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fd61e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fd61e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55640/0 (socket says 172.21.15.49:55640) 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fd61e640 1 -- 172.21.15.49:0/3121785610 learned_addr learned my addr 172.21.15.49:0/3121785610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fde1f640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9f80c6d50 0x7fa9f81598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 0x7fa9f80b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 -- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9f80c6d50 msgr2=0x7fa9f81598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9f80c6d50 0x7fa9f81598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 -- 172.21.15.49:0/3121785610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 msgr2=0x7fa9f8159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 -- 172.21.15.49:0/3121785610 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f81f0690 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fd61e640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.456+0000 7fa9fe620640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 0x7fa9f80b55b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000cab0 tx=0x7fa9f000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.457+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0017070 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.457+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f000ac30 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.457+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f80b4ad0 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.457+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0012600 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.457+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f81589b0 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.458+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f80b48c0 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.458+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9f001a070 con 0x7fa9f8154f70 2024-05-09T08:56:56.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.459+0000 7fa9ee7fc640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 0x7fa9dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.459+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(29..29 src has 1..29) ==== 5857+0+0 (secure 0 0 0) 0x7fa9f008ac60 con 0x7fa9f8154f70 2024-05-09T08:56:56.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.459+0000 7fa9fde1f640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 0x7fa9dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.460+0000 7fa9fde1f640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 0x7fa9dc06e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.463+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9f000ae20 con 0x7fa9f8154f70 2024-05-09T08:56:56.473 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.473 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.483 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:56:56.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.494+0000 7f4160a9e640 1 Processor -- start 2024-05-09T08:56:56.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.494+0000 7f4160a9e640 1 -- start start 2024-05-09T08:56:56.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.495+0000 7f4160a9e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 0x7f415c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.495+0000 7f4160a9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4160a9e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415c072600 con 0x7f415c0c6640 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4160a9e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f415c0728e0 con 0x7f415c154b90 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4160a9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f415c0c5880 con 0x7f415c154f70 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f415a59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 0x7f415c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4159d9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f415ad9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f415c0c6640 0x7f415c0c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51200/0 (socket says 172.21.15.49:51200) 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f415ad9d640 1 -- 172.21.15.49:0/786910114 learned_addr learned my addr 172.21.15.49:0/786910114 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4159d9b640 1 -- 172.21.15.49:0/786910114 >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 msgr2=0x7f415c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4159d9b640 1 --2- 172.21.15.49:0/786910114 >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 0x7f415c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4159d9b640 1 -- 172.21.15.49:0/786910114 >> v1:172.21.15.49:6789/0 conn(0x7f415c0c6640 legacy=0x7f415c0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f4159d9b640 1 -- 172.21.15.49:0/786910114 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c0c5b60 con 0x7f415c154f70 2024-05-09T08:56:56.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.496+0000 7f415a59c640 1 --2- 172.21.15.49:0/786910114 >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 0x7f415c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:56.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.497+0000 7f4159d9b640 1 --2- 172.21.15.49:0/786910114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c0cb7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41500097a0 tx=0x7f415002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2113154a539335b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.497+0000 7f415959a640 1 -- 172.21.15.49:0/786910114 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150037070 con 0x7f415c154f70 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.497+0000 7f415959a640 1 -- 172.21.15.49:0/786910114 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4150032440 con 0x7f415c154f70 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.497+0000 7f415959a640 1 -- 172.21.15.49:0/786910114 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150032810 con 0x7f415c154f70 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 -- 172.21.15.49:0/786910114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 msgr2=0x7f415c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 --2- 172.21.15.49:0/786910114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c0cb7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41500097a0 tx=0x7f415002a210 comp rx=0 tx=0).stop 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 -- 172.21.15.49:0/786910114 shutdown_connections 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 --2- 172.21.15.49:0/786910114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c0cb7c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 --2- 172.21.15.49:0/786910114 >> v2:172.21.15.49:3300/0 conn(0x7f415c154b90 0x7f415c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 -- 172.21.15.49:0/786910114 >> 172.21.15.49:0/786910114 conn(0x7f415c074840 msgr2=0x7f415c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 -- 172.21.15.49:0/786910114 shutdown_connections 2024-05-09T08:56:56.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.498+0000 7f4160a9e640 1 -- 172.21.15.49:0/786910114 wait complete. 2024-05-09T08:56:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.500+0000 7f4160a9e640 1 Processor -- start 2024-05-09T08:56:56.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.500+0000 7f4160a9e640 1 -- start start 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f415c154b90 0x7f415c0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f415c15af40 con 0x7f415c0c6640 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f415c0c6120 con 0x7f415c154b90 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.501+0000 7f4160a9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f415c071820 con 0x7f415c154f70 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55656/0 (socket says 172.21.15.49:55656) 2024-05-09T08:56:56.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 -- 172.21.15.49:0/1918605915 learned_addr learned my addr 172.21.15.49:0/1918605915 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f4159d9b640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f415c154b90 0x7f415c0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415ad9d640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 -- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 msgr2=0x7f415c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 -- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f415c154b90 msgr2=0x7f415c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f415c154b90 0x7f415c0cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 -- 172.21.15.49:0/1918605915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c1f14e0 con 0x7f415c0c6640 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f415a59c640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f414c00ec10 tx=0x7f414c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c017070 con 0x7f415c0c6640 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414c00cc30 con 0x7f415c0c6640 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c1f1750 con 0x7f415c0c6640 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c013470 con 0x7f415c0c6640 2024-05-09T08:56:56.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.502+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c147a00 con 0x7f415c0c6640 2024-05-09T08:56:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.503+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c04f660 con 0x7f415c0c6640 2024-05-09T08:56:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.504+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f414c01a070 con 0x7f415c0c6640 2024-05-09T08:56:56.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.505+0000 7f414affd640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 0x7f413c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.505+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5857+0+0 (secure 0 0 0) 0x7f414c08aeb0 con 0x7f415c0c6640 2024-05-09T08:56:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.505+0000 7f4159d9b640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 0x7f413c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:56.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.506+0000 7f4159d9b640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 0x7f413c06e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4150034000 tx=0x7f415002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:56.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.509+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f414c0597a0 con 0x7f415c0c6640 2024-05-09T08:56:56.569 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:56.569+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:56.625 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.625 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:56:56.655 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:56.657+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:56.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.660+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fa9f804f660 con 0x7fa9f8154f70 2024-05-09T08:56:56.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:56.708+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f415c0c6a90 con 0x7f415c0c6640 2024-05-09T08:56:56.756 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:56.756+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:56.778 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.778 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.787 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:56:56.857 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:56.858+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:56.911 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:56.911 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:56.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:56:56.964 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:56.965+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:57.032 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.032 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:56:57.082 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:57.083+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:57.160 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:57.161+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:57.193 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.193 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:56:57.261 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:57.261+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:57.313 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.313 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:56:57.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.336+0000 7f414affd640 1 -- 172.21.15.49:0/1918605915 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v30) ==== 113+0+0 (secure 0 0 0) 0x7f414c0590a0 con 0x7f415c0c6640 2024-05-09T08:56:57.335 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.338+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 msgr2=0x7f413c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.338+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 0x7f413c06e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4150034000 tx=0x7f415002e000 comp rx=0 tx=0).stop 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.338+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 msgr2=0x7f415c1514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.338+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f414c00ec10 tx=0x7f414c00c520 comp rx=0 tx=0).stop 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 shutdown_connections 2024-05-09T08:56:57.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f413c06be30 0x7f413c06e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f415c154f70 0x7f415c14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f415c154b90 0x7f415c0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 --2- 172.21.15.49:0/1918605915 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f415c0c6640 0x7f415c1514a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 >> 172.21.15.49:0/1918605915 conn(0x7f415c074840 msgr2=0x7f415c15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 shutdown_connections 2024-05-09T08:56:57.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7f4160a9e640 1 -- 172.21.15.49:0/1918605915 wait complete. 2024-05-09T08:56:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.339+0000 7fa9ee7fc640 1 -- 172.21.15.49:0/3121785610 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v30) ==== 97+0+0 (secure 0 0 0) 0x7fa9f0058df0 con 0x7fa9f8154f70 2024-05-09T08:56:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:56:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 msgr2=0x7fa9dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 0x7fa9dc06e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4034000 tx=0x7fa9f402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 msgr2=0x7fa9f80b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 0x7fa9f80b55b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000cab0 tx=0x7fa9f000cf70 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 shutdown_connections 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa9dc06be30 0x7fa9dc06e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9f8154f70 0x7fa9f80b55b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa9f8154b90 0x7fa9f8159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.342+0000 7faa00083640 1 --2- 172.21.15.49:0/3121785610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9f80c6d50 0x7fa9f81598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.343+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 >> 172.21.15.49:0/3121785610 conn(0x7fa9f8074840 msgr2=0x7fa9f814f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.343+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 shutdown_connections 2024-05-09T08:56:57.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.343+0000 7faa00083640 1 -- 172.21.15.49:0/3121785610 wait complete. 2024-05-09T08:56:57.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-09T08:56:57.401 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:57.402+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:57.436 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.436 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.447 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:56:57.488 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:56:57.489+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:56:57.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.506+0000 7f1e63838640 1 Processor -- start 2024-05-09T08:56:57.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 -- start start 2024-05-09T08:56:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 0x7f1e5c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c0c6fd0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.507+0000 7f1e63838640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1e5c072600 con 0x7f1e5c154b90 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e63838640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c0728e0 con 0x7f1e5c154f70 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55658/0 (socket says 172.21.15.49:55658) 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 -- 172.21.15.49:0/498203038 learned_addr learned my addr 172.21.15.49:0/498203038 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e615d4640 1 --2- 172.21.15.49:0/498203038 >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 0x7f1e5c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e60dd3640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 -- 172.21.15.49:0/498203038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 msgr2=0x7f1e5c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 -- 172.21.15.49:0/498203038 >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 msgr2=0x7f1e5c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:57.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 --2- 172.21.15.49:0/498203038 >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 0x7f1e5c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e61dd5640 1 -- 172.21.15.49:0/498203038 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c0c5880 con 0x7f1e5c0c6640 2024-05-09T08:56:57.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.508+0000 7f1e615d4640 1 --2- 172.21.15.49:0/498203038 >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 0x7f1e5c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.509+0000 7f1e61dd5640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cdf50 tx=0x7f1e5402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fb79a9bdf49a66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.509+0000 7f1e53fff640 1 -- 172.21.15.49:0/498203038 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54036070 con 0x7f1e5c0c6640 2024-05-09T08:56:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.509+0000 7f1e53fff640 1 -- 172.21.15.49:0/498203038 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e54031600 con 0x7f1e5c0c6640 2024-05-09T08:56:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.509+0000 7f1e53fff640 1 -- 172.21.15.49:0/498203038 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e540319d0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 -- 172.21.15.49:0/498203038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 msgr2=0x7f1e5c0c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0cdf50 tx=0x7f1e5402a730 comp rx=0 tx=0).stop 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 -- 172.21.15.49:0/498203038 shutdown_connections 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c0c6a90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 --2- 172.21.15.49:0/498203038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 --2- 172.21.15.49:0/498203038 >> v2:172.21.15.183:3300/0 conn(0x7f1e5c154b90 0x7f1e5c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 -- 172.21.15.49:0/498203038 >> 172.21.15.49:0/498203038 conn(0x7f1e5c074840 msgr2=0x7f1e5c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 -- 172.21.15.49:0/498203038 shutdown_connections 2024-05-09T08:56:57.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.510+0000 7f1e63838640 1 -- 172.21.15.49:0/498203038 wait complete. 2024-05-09T08:56:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.512+0000 7f1e63838640 1 Processor -- start 2024-05-09T08:56:57.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.512+0000 7f1e63838640 1 -- start start 2024-05-09T08:56:57.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5c154b90 0x7f1e5c144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0c6120 con 0x7f1e5c0c6640 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c0c5e40 con 0x7f1e5c154b90 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e63838640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5c15af40 con 0x7f1e5c154f70 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e615d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e615d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55674/0 (socket says 172.21.15.49:55674) 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.513+0000 7f1e615d4640 1 -- 172.21.15.49:0/4075789948 learned_addr learned my addr 172.21.15.49:0/4075789948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e61dd5640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e60dd3640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5c154b90 0x7f1e5c144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 -- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 msgr2=0x7f1e5c145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 -- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5c154b90 msgr2=0x7f1e5c144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5c154b90 0x7f1e5c144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 -- 172.21.15.49:0/4075789948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1596d0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e615d4640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800d7d0 tx=0x7f1e5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e58004030 con 0x7f1e5c0c6640 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e580041d0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1599b0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800b6c0 con 0x7f1e5c0c6640 2024-05-09T08:56:57.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.514+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c159f70 con 0x7f1e5c0c6640 2024-05-09T08:56:57.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.515+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c04f660 con 0x7f1e5c0c6640 2024-05-09T08:56:57.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.516+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e5800b860 con 0x7f1e5c0c6640 2024-05-09T08:56:57.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.517+0000 7f1e51ffb640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 0x7f1e4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:57.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.517+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5857+0+0 (secure 0 0 0) 0x7f1e5808b220 con 0x7f1e5c0c6640 2024-05-09T08:56:57.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.517+0000 7f1e60dd3640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 0x7f1e4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:57.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.518+0000 7f1e60dd3640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 0x7f1e4006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c154470 tx=0x7f1e4c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:57.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.521+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e58059b20 con 0x7f1e5c0c6640 2024-05-09T08:56:57.563 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:57.564+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:57.600 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.600 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:56:57.696 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:57.697+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:57.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:57.719+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f1e5c158920 con 0x7f1e5c0c6640 2024-05-09T08:56:57.765 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.765 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.765 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:57.766+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:57.773 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:56:57.866 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:57.866+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:57.924 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:57.924 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:57.933 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:56:57.976 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:57.977+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:58.042 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:58.042 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:58.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:56:58.092 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:58.093+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:58.168 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:58.169+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:58.201 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:58.201 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:58.209 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:56:58.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:56:58.294 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:58.295+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.337+0000 7f1e51ffb640 1 -- 172.21.15.49:0/4075789948 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v31) ==== 113+0+0 (secure 0 0 0) 0x7f1e58059420 con 0x7f1e5c0c6640 2024-05-09T08:56:58.337 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-05-09T08:56:58.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.340+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 msgr2=0x7f1e4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.340+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 0x7f1e4006e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c154470 tx=0x7f1e4c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.340+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 msgr2=0x7f1e5c15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.340+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800d7d0 tx=0x7f1e5800dc90 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.340+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 shutdown_connections 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e4006be30 0x7f1e4006e2f0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c154470 tx=0x7f1e4c008040 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e5c154f70 0x7f1e5c145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e5c154b90 0x7f1e5c144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 --2- 172.21.15.49:0/4075789948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e5c0c6640 0x7f1e5c15a9a0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1e5800d7d0 tx=0x7f1e5800dc90 comp rx=0 tx=0).stop 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 >> 172.21.15.49:0/4075789948 conn(0x7f1e5c074840 msgr2=0x7f1e5c152180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:58.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 shutdown_connections 2024-05-09T08:56:58.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.341+0000 7f1e63838640 1 -- 172.21.15.49:0/4075789948 wait complete. 2024-05-09T08:56:58.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-09T08:56:58.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.357+0000 7f366179e640 1 Processor -- start 2024-05-09T08:56:58.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.357+0000 7f366179e640 1 -- start start 2024-05-09T08:56:58.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f366179e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f365c15b410 0x7f365c067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f366179e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c0c91f0 con 0x7f365c073360 2024-05-09T08:56:58.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f366179e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c0c8f10 con 0x7f365c15b030 2024-05-09T08:56:58.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f366179e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f365c06ef20 con 0x7f365c15b410 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f365b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f365c073360 0x7f365c0c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51216/0 (socket says 172.21.15.49:51216) 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f365a7fc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f365c15b410 0x7f365c067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.358+0000 7f365b7fe640 1 -- 172.21.15.49:0/742446083 learned_addr learned my addr 172.21.15.49:0/742446083 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.359+0000 7f365a7fc640 1 -- 172.21.15.49:0/742446083 >> v1:172.21.15.183:6789/0 conn(0x7f365c15b030 legacy=0x7f365c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.359+0000 7f3659ffb640 1 -- 172.21.15.49:0/742446083 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578221324 0 0) 0x7f365c0c91f0 con 0x7f365c073360 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.359+0000 7f365a7fc640 1 -- 172.21.15.49:0/742446083 >> v1:172.21.15.49:6789/0 conn(0x7f365c073360 legacy=0x7f365c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.359+0000 7f365a7fc640 1 -- 172.21.15.49:0/742446083 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c15b8f0 con 0x7f365c15b410 2024-05-09T08:56:58.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.359+0000 7f365a7fc640 1 --2- 172.21.15.49:0/742446083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c067a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36500097a0 tx=0x7f365002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb174cc3a12e54c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f3659ffb640 1 -- 172.21.15.49:0/742446083 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650037070 con 0x7f365c15b410 2024-05-09T08:56:58.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f3659ffb640 1 -- 172.21.15.49:0/742446083 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3650032640 con 0x7f365c15b410 2024-05-09T08:56:58.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f3659ffb640 1 -- 172.21.15.49:0/742446083 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3650032a10 con 0x7f365c15b410 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 -- 172.21.15.49:0/742446083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 msgr2=0x7f365c067a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 --2- 172.21.15.49:0/742446083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c067a80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36500097a0 tx=0x7f365002a2c0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 -- 172.21.15.49:0/742446083 shutdown_connections 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 --2- 172.21.15.49:0/742446083 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c067a80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 -- 172.21.15.49:0/742446083 >> 172.21.15.49:0/742446083 conn(0x7f365c0c9440 msgr2=0x7f365c0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 -- 172.21.15.49:0/742446083 shutdown_connections 2024-05-09T08:56:58.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.360+0000 7f366179e640 1 -- 172.21.15.49:0/742446083 wait complete. 2024-05-09T08:56:58.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.362+0000 7f366179e640 1 Processor -- start 2024-05-09T08:56:58.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.363+0000 7f366179e640 1 -- start start 2024-05-09T08:56:58.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f365c073360 0x7f365c0cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365c15b030 0x7f365c1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1552f0 con 0x7f365c15b410 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f365c06ebb0 con 0x7f365c15b030 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f366179e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f365c155470 con 0x7f365c073360 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55684/0 (socket says 172.21.15.49:55684) 2024-05-09T08:56:58.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 learned_addr learned my addr 172.21.15.49:0/2676719698 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365c15b030 0x7f365c1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f365c073360 0x7f365c0cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f365c073360 msgr2=0x7f365c0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f365c073360 0x7f365c0cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365c15b030 msgr2=0x7f365c1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365c15b030 0x7f365c1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.364+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c151180 con 0x7f365c15b410 2024-05-09T08:56:58.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f365400c780 tx=0x7f365400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654018070 con 0x7f365c15b410 2024-05-09T08:56:58.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365400ce30 con 0x7f365c15b410 2024-05-09T08:56:58.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c151460 con 0x7f365c15b410 2024-05-09T08:56:58.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400a840 con 0x7f365c15b410 2024-05-09T08:56:58.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.365+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c14e100 con 0x7f365c15b410 2024-05-09T08:56:58.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.366+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f365c067620 con 0x7f365c15b410 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.367+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f365401b070 con 0x7f365c15b410 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.367+0000 7f36437fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 0x7f364806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.367+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5857+0+0 (secure 0 0 0) 0x7f365408a450 con 0x7f365c15b410 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.367+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v31) ==== 54+0+16 (secure 0 0 0) 0x7f365c067620 con 0x7f365c15b410 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.368+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 0x7f364806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.368+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f365c150ac0 0x7f365c14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.368+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f365c0c3840 con 0x7f365c150ac0 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.368+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f365c150ac0 0x7f365c14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.368+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 0x7f364806e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f364c000f60 tx=0x7f364c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.369+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f365c150ac0 0x7f365c14f0f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.369+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c0c3840 con 0x7f365c150ac0 2024-05-09T08:56:58.370 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:56:58.370 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:56:58.371 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:56:58.371+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:56:58.379 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:56:58.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.385+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c067860 con 0x7f365c150ac0 2024-05-09T08:56:58.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:56:58.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.387+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c067860 con 0x7f365c150ac0 2024-05-09T08:56:58.387 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:56:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.387+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f365c14fb40 0x7f365c144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.387+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f365c14ff20 con 0x7f365c14fb40 2024-05-09T08:56:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.387+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f365c14fb40 0x7f365c144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.388+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f365c14fb40 0x7f365c144d90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.388+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c14ff20 con 0x7f365c14fb40 2024-05-09T08:56:58.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.406+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c0719b0 con 0x7f365c14fb40 2024-05-09T08:56:58.407 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.408+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c0719b0 con 0x7f365c14fb40 2024-05-09T08:56:58.407 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.408+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f365c1457e0 0x7f365c061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.408+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f365c0cc280 con 0x7f365c1457e0 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.408+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f365c1457e0 0x7f365c061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.408+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f365c1457e0 0x7f365c061c70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.409+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c0cc280 con 0x7f365c1457e0 2024-05-09T08:56:58.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.424+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c0621b0 con 0x7f365c1457e0 2024-05-09T08:56:58.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.426+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c0621b0 con 0x7f365c1457e0 2024-05-09T08:56:58.425 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:56:58.425 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:56:58.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.426+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f365c02c040 0x7f365c02c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.426+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f365c062ac0 con 0x7f365c02c040 2024-05-09T08:56:58.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.426+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f365c02c040 0x7f365c02c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.426+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f365c02c040 0x7f365c02c460 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.427+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c062ac0 con 0x7f365c02c040 2024-05-09T08:56:58.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.442+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c02cc70 con 0x7f365c02c040 2024-05-09T08:56:58.442 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.443+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c02cc70 con 0x7f365c02c040 2024-05-09T08:56:58.443 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:56:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:56:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.444+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f365c0c6980 0x7f365c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.444+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f365c02d390 con 0x7f365c0c6980 2024-05-09T08:56:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.444+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f365c0c6980 0x7f365c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.445+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f365c0c6980 0x7f365c0c6dc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.446+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c02d390 con 0x7f365c0c6980 2024-05-09T08:56:58.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.461+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c02d390 con 0x7f365c0c6980 2024-05-09T08:56:58.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.463+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c02d390 con 0x7f365c0c6980 2024-05-09T08:56:58.462 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:56:58.463 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:56:58.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.463+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f365c0c7810 0x7f365c2078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.463+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f365c02d390 con 0x7f365c0c7810 2024-05-09T08:56:58.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.464+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f365c0c7810 0x7f365c2078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.464+0000 7f365a7fc640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f365c0c7810 0x7f365c2078a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.465+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c02d390 con 0x7f365c0c7810 2024-05-09T08:56:58.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.467+0000 7fd266bce640 1 Processor -- start 2024-05-09T08:56:58.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.467+0000 7fd266bce640 1 -- start start 2024-05-09T08:56:58.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.468+0000 7fd266bce640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd2601512d0 0x7fd26006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.468+0000 7fd266bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd26015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.468+0000 7fd266bce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd260158000 con 0x7fd26006b9f0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.468+0000 7fd266bce640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd260157480 con 0x7fd2601512d0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.468+0000 7fd266bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd260157760 con 0x7fd26006c690 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd25ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd26006b9f0 0x7fd26006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51224/0 (socket says 172.21.15.49:51224) 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd25ffff640 1 -- 172.21.15.49:0/3417415053 learned_addr learned my addr 172.21.15.49:0/3417415053 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26496a640 1 --2- 172.21.15.49:0/3417415053 >> v2:172.21.15.183:3300/0 conn(0x7fd2601512d0 0x7fd26006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26516b640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd26015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd25f7fe640 1 -- 172.21.15.49:0/3417415053 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3871732857 0 0) 0x7fd260158000 con 0x7fd26006b9f0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd25f7fe640 1 -- 172.21.15.49:0/3417415053 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd250003520 con 0x7fd26006b9f0 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26496a640 1 -- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 msgr2=0x7fd26015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26496a640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd26015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26496a640 1 -- 172.21.15.49:0/3417415053 >> v1:172.21.15.49:6789/0 conn(0x7fd26006b9f0 legacy=0x7fd26006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.469+0000 7fd26496a640 1 -- 172.21.15.49:0/3417415053 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd260157a40 con 0x7fd2601512d0 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.470+0000 7fd26496a640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2601512d0 0x7fd26006b4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2540097a0 tx=0x7fd25402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82375c29782c319c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.470+0000 7fd25f7fe640 1 -- 172.21.15.49:0/3417415053 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254037070 con 0x7fd2601512d0 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.470+0000 7fd25f7fe640 1 -- 172.21.15.49:0/3417415053 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd254032440 con 0x7fd2601512d0 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.470+0000 7fd25f7fe640 1 -- 172.21.15.49:0/3417415053 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254032810 con 0x7fd2601512d0 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 -- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2601512d0 msgr2=0x7fd26006b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2601512d0 0x7fd26006b4b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2540097a0 tx=0x7fd25402a210 comp rx=0 tx=0).stop 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 -- 172.21.15.49:0/3417415053 shutdown_connections 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd26015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 --2- 172.21.15.49:0/3417415053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2601512d0 0x7fd26006b4b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 -- 172.21.15.49:0/3417415053 >> 172.21.15.49:0/3417415053 conn(0x7fd2600cc290 msgr2=0x7fd2600cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 -- 172.21.15.49:0/3417415053 shutdown_connections 2024-05-09T08:56:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.471+0000 7fd266bce640 1 -- 172.21.15.49:0/3417415053 wait complete. 2024-05-09T08:56:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.473+0000 7fd266bce640 1 Processor -- start 2024-05-09T08:56:58.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.473+0000 7fd266bce640 1 -- start start 2024-05-09T08:56:58.473 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:58.473+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 0x7fd260162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd260144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2601570b0 con 0x7fd2601512d0 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2600c8af0 con 0x7fd26006b9f0 2024-05-09T08:56:58.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd266bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd260157230 con 0x7fd26006c690 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55696/0 (socket says 172.21.15.49:55696) 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 -- 172.21.15.49:0/1899584601 learned_addr learned my addr 172.21.15.49:0/1899584601 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26496a640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 0x7fd260162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd25ffff640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd260144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 -- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 msgr2=0x7fd260144c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd260144c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 -- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 msgr2=0x7fd260162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 0x7fd260162cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 -- 172.21.15.49:0/1899584601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd260147e00 con 0x7fd2601512d0 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26496a640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 0x7fd260162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd26516b640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd258009dd0 tx=0x7fd25800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.475+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258018020 con 0x7fd2601512d0 2024-05-09T08:56:58.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.476+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258014480 con 0x7fd2601512d0 2024-05-09T08:56:58.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.476+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd260150750 con 0x7fd2601512d0 2024-05-09T08:56:58.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.476+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258014870 con 0x7fd2601512d0 2024-05-09T08:56:58.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.476+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26014e530 con 0x7fd2601512d0 2024-05-09T08:56:58.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.477+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26004f660 con 0x7fd2601512d0 2024-05-09T08:56:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.478+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd258005d10 con 0x7fd2601512d0 2024-05-09T08:56:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.478+0000 7fd25d7fa640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 0x7fd24406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.478+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5857+0+0 (secure 0 0 0) 0x7fd25808be20 con 0x7fd2601512d0 2024-05-09T08:56:58.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.479+0000 7fd26496a640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 0x7fd24406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.480+0000 7fd26496a640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 0x7fd24406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd254034000 tx=0x7fd25402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.483+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd258022030 con 0x7fd2601512d0 2024-05-09T08:56:58.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.484+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c02d390 con 0x7f365c0c7810 2024-05-09T08:56:58.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.486+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c02d390 con 0x7f365c0c7810 2024-05-09T08:56:58.485 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:56:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:56:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.486+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f365c0c7ca0 0x7f365c20a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.486+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f365c20d0d0 con 0x7f365c0c7ca0 2024-05-09T08:56:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.487+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f365c0c7ca0 0x7f365c20a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.487+0000 7f365b7fe640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f365c0c7ca0 0x7f365c20a300 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.488+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c20d0d0 con 0x7f365c0c7ca0 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.819653958000004, 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:56:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.819633987000003, 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:56:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.818805220999998, 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:56:58.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.817802329000003, 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.514 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.817499497999997, 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:56:58.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:56:58.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.776312939999997, 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.523 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:56:58.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:56:58.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:58.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.528 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.776300913999997, 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:56:58.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.776287443999998, 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.536 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.775263041999999, 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:56:58.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.774835752999998, 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:56:58.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.774719413, 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:58.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.549 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.773445416999998, 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:56:58.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:56:58.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.771696016999996, 2024-05-09T08:56:58.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:56:58.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.771367873000003, 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:56:58.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.761239244999999, 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:56:58.562 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:56:58.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:56:58.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:58.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 40.770496244, 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:56:58.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:56:58.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:56:58.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:56:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 40.770486857999998, 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:56:58.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.581 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 40.770475814000001, 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:56:58.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 40.768740817999998, 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:56:58.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.738836612, 2024-05-09T08:56:58.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:56:58.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:56:58.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:56:58.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:56:58.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:56:58.596 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:56:58.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.503+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c20a840 con 0x7f365c0c7ca0 2024-05-09T08:56:58.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.505+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c20a840 con 0x7f365c0c7ca0 2024-05-09T08:56:58.597 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:56:58.598 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:56:58.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.505+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f365c20aec0 0x7f365c20d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:58.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.505+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f365c210140 con 0x7f365c20aec0 2024-05-09T08:56:58.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.505+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f365c20aec0 0x7f365c20d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.506+0000 7f365affd640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f365c20aec0 0x7f365c20d2b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:58.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.507+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f365c210140 con 0x7f365c20aec0 2024-05-09T08:56:58.600 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:56:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.521+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f365c20d7f0 con 0x7f365c20aec0 2024-05-09T08:56:58.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.523+0000 7f36437fe640 1 -- 172.21.15.49:0/2676719698 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f365c20d7f0 con 0x7f365c20aec0 2024-05-09T08:56:58.600 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:56:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:56:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f365c150ac0 msgr2=0x7f365c14f0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f365c150ac0 0x7f365c14f0f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f365c14fb40 msgr2=0x7f365c144d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f365c14fb40 0x7f365c144d90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f365c1457e0 msgr2=0x7f365c061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f365c1457e0 0x7f365c061c70 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f365c02c040 msgr2=0x7f365c02c460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f365c02c040 0x7f365c02c460 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f365c0c6980 msgr2=0x7f365c0c6dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f365c0c6980 0x7f365c0c6dc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 reap_dead start 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f365c0c7810 msgr2=0x7f365c2078a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f365c0c7810 0x7f365c2078a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 reap_dead start 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f365c0c7ca0 msgr2=0x7f365c20a300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f365c0c7ca0 0x7f365c20a300 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f365c20aec0 msgr2=0x7f365c20d2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f365c20aec0 0x7f365c20d2b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f365b7fe640 1 -- 172.21.15.49:0/2676719698 reap_dead start 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 msgr2=0x7f364806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 0x7f364806e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f364c000f60 tx=0x7f364c009040 comp rx=0 tx=0).stop 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 msgr2=0x7f365c1f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f365400c780 tx=0x7f365400cc40 comp rx=0 tx=0).stop 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 shutdown_connections 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f364806be60 0x7f364806e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 --2- 172.21.15.49:0/2676719698 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f365c15b410 0x7f365c1f15c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 >> 172.21.15.49:0/2676719698 conn(0x7f365c0c9440 msgr2=0x7f365c06df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 shutdown_connections 2024-05-09T08:56:58.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.524+0000 7f366179e640 1 -- 172.21.15.49:0/2676719698 wait complete. 2024-05-09T08:56:58.605 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:58.597+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:58.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:58.680+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7fd26006cac0 con 0x7fd2601512d0 2024-05-09T08:56:58.699 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:58.699+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:58.716 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:58.716 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:58.717 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:58.717 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:58.726 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:56:58.810 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:58.810+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:58.868 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:58.868 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:58.869 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:58.869 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:58.869 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:58.877 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:56:58.900 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:56:58.901+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:56:59.001 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:56:59.001+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.022 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.342224420000001, 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:56:59.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.338804658000001, 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:59.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:56:59.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:56:59.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:59.033 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.338578149, 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:56:59.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.338524855999999, 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:56:59.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:56:59.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:56:59.042 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.290201285000002, 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:56:59.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:59.047 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.289675410000001, 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:56:59.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.051 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.289428078, 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:56:59.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.283988874999999, 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:56:59.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:56:59.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:59.061 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.276029711, 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:56:59.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.275465531999998, 2024-05-09T08:56:59.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:56:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:56:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.275378459000002, 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:56:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.275309401000001, 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:56:59.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.079 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.273951091999997, 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:56:59.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.084 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.273905845000002, 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:56:59.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:56:59.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.273650578000002, 2024-05-09T08:56:59.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:56:59.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.273611881999997, 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.094 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:56:59.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.260668824, 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:56:59.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.103 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.260029876000001, 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:56:59.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:59.107 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.259475932000001, 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.111 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.256621794000001, 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.116 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.117 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:56:59.118 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:59.118+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:59.217 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:59.217+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:59.262 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.262 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:59.262 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:59.262 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.271 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:56:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.346+0000 7fd25d7fa640 1 -- 172.21.15.49:0/1899584601 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v32) ==== 113+0+0 (secure 0 0 0) 0x7fd258010040 con 0x7fd2601512d0 2024-05-09T08:56:59.346 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-05-09T08:56:59.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.349+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 msgr2=0x7fd24406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.349+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 0x7fd24406e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd254034000 tx=0x7fd25402e000 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.349+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 msgr2=0x7fd2601451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.349+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd258009dd0 tx=0x7fd25800c490 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 shutdown_connections 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd24406be30 0x7fd24406e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2601512d0 0x7fd2601451a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd26006c690 0x7fd260144c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 --2- 172.21.15.49:0/1899584601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd26006b9f0 0x7fd260162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 >> 172.21.15.49:0/1899584601 conn(0x7fd2600cc290 msgr2=0x7fd2600c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:59.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 shutdown_connections 2024-05-09T08:56:59.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.350+0000 7fd266bce640 1 -- 172.21.15.49:0/1899584601 wait complete. 2024-05-09T08:56:59.356 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:56:59.356+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:56:59.359 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:56:59.362 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-09T08:56:59.416 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.416 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:59.416 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:59.416 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:59.416 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.419 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:56:59.419+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:56:59.425 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:56:59.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.471+0000 7f1ca0f14640 1 Processor -- start 2024-05-09T08:56:59.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.471+0000 7f1ca0f14640 1 -- start start 2024-05-09T08:56:59.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1ca0f14640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 0x7f1c9c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1ca0f14640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1c9c0c6d50 0x7f1c9c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1ca0f14640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9c072600 con 0x7f1c9c154f70 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1ca0f14640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c0728e0 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1ca0f14640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0c5880 con 0x7f1c9c154b90 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1c99d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c9c154f70 0x7f1c9c0c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51240/0 (socket says 172.21.15.49:51240) 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1c99d9b640 1 -- 172.21.15.49:0/3033434765 learned_addr learned my addr 172.21.15.49:0/3033434765 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1c9a59c640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 0x7f1c9c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.472+0000 7f1c9ad9d640 1 --2- 172.21.15.49:0/3033434765 >> v2:172.21.15.183:3300/0 conn(0x7f1c9c0c6d50 0x7f1c9c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9959a640 1 -- 172.21.15.49:0/3033434765 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1294861655 0 0) 0x7f1c9c072600 con 0x7f1c9c154f70 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9959a640 1 -- 172.21.15.49:0/3033434765 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c88003520 con 0x7f1c9c154f70 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9ad9d640 1 -- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 msgr2=0x7f1c9c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:59.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9ad9d640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 0x7f1c9c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9ad9d640 1 -- 172.21.15.49:0/3033434765 >> v1:172.21.15.49:6789/0 conn(0x7f1c9c154f70 legacy=0x7f1c9c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9ad9d640 1 -- 172.21.15.49:0/3033434765 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c0c5b60 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.473+0000 7f1c9a59c640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 0x7f1c9c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:59.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1c9ad9d640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c0c6d50 0x7f1c9c0cb7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097a0 tx=0x7f1c9402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3b46da28655ed5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1c9959a640 1 -- 172.21.15.49:0/3033434765 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94037070 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1c9959a640 1 -- 172.21.15.49:0/3033434765 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94032440 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1c9959a640 1 -- 172.21.15.49:0/3033434765 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c940327f0 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1ca0f14640 1 -- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c0c6d50 msgr2=0x7f1c9c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.474+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c0c6d50 0x7f1c9c0cb7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c940097a0 tx=0x7f1c9402a210 comp rx=0 tx=0).stop 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 -- 172.21.15.49:0/3033434765 shutdown_connections 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c0c6d50 0x7f1c9c0cb7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/3033434765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c154b90 0x7f1c9c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 -- 172.21.15.49:0/3033434765 >> 172.21.15.49:0/3033434765 conn(0x7f1c9c074840 msgr2=0x7f1c9c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 -- 172.21.15.49:0/3033434765 shutdown_connections 2024-05-09T08:56:59.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.475+0000 7f1ca0f14640 1 -- 172.21.15.49:0/3033434765 wait complete. 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.476+0000 7f1ca0f14640 1 Processor -- start 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.476+0000 7f1ca0f14640 1 -- start start 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 0x7f1c9c14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 0x7f1c9c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c15af40 con 0x7f1c9c154f70 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0c6120 con 0x7f1c9c154b90 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1ca0f14640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c071820 con 0x7f1c9c0c6d50 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 0x7f1c9c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9a59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 0x7f1c9c14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9ad9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55700/0 (socket says 172.21.15.49:55700) 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9ad9d640 1 -- 172.21.15.49:0/4028152652 learned_addr learned my addr 172.21.15.49:0/4028152652 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 -- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 msgr2=0x7f1c9c14aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 0x7f1c9c14aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 -- 172.21.15.49:0/4028152652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 msgr2=0x7f1c9c14c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 -- 172.21.15.49:0/4028152652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c150840 con 0x7f1c9c154b90 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9ad9d640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c9a59c640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 0x7f1c9c14aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:59.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.477+0000 7f1c99d9b640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 0x7f1c9c14a3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c3c0 tx=0x7f1c84001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.478+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84017070 con 0x7f1c9c154b90 2024-05-09T08:56:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.478+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84005ce0 con 0x7f1c9c154b90 2024-05-09T08:56:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.478+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84007310 con 0x7f1c9c154b90 2024-05-09T08:56:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.478+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c151e90 con 0x7f1c9c154b90 2024-05-09T08:56:59.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.478+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1f42a0 con 0x7f1c9c154b90 2024-05-09T08:56:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.479+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c04f660 con 0x7f1c9c154b90 2024-05-09T08:56:59.482 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.480+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c8401a070 con 0x7f1c9c154b90 2024-05-09T08:56:59.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.480+0000 7f1c92ffd640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 0x7f1c7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.480+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5857+0+0 (secure 0 0 0) 0x7f1c8408a1d0 con 0x7f1c9c154b90 2024-05-09T08:56:59.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.480+0000 7f1c9a59c640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 0x7f1c7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.481+0000 7f1c9a59c640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 0x7f1c7c06e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fd0 tx=0x7f1c8c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.485+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c84058400 con 0x7f1c9c154b90 2024-05-09T08:56:59.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.513+0000 7f7f31279640 1 Processor -- start 2024-05-09T08:56:59.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.513+0000 7f7f31279640 1 -- start start 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f31279640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f31279640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f2c0c6b80 0x7f7f2c071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f31279640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c0c6fd0 con 0x7f7f2c154b90 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f31279640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c072600 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f31279640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0728e0 con 0x7f7f2c154f70 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f2a7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f2b7fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f2c0c6b80 0x7f7f2c071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f2a7fc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:35038/0 (socket says 172.21.15.49:35038) 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f2affd640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f7f2c154b90 0x7f7f2c0cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49606/0 (socket says 172.21.15.49:49606) 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.514+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/477754043 learned_addr learned my addr 172.21.15.49:0/477754043 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.515+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/477754043 >> v2:172.21.15.183:3300/0 conn(0x7f7f2c0c6b80 msgr2=0x7f7f2c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.515+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/477754043 >> v2:172.21.15.183:3300/0 conn(0x7f7f2c0c6b80 0x7f7f2c071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.515+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/477754043 >> v1:172.21.15.183:6789/0 conn(0x7f7f2c154b90 legacy=0x7f7f2c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.515+0000 7f7f2a7fc640 1 -- 172.21.15.49:0/477754043 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c0c5880 con 0x7f7f2c154f70 2024-05-09T08:56:59.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.515+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/477754043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf66830ec8f05c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f29ffb640 1 -- 172.21.15.49:0/477754043 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20037070 con 0x7f7f2c154f70 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f29ffb640 1 -- 172.21.15.49:0/477754043 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2002ae50 con 0x7f7f2c154f70 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f29ffb640 1 -- 172.21.15.49:0/477754043 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20032860 con 0x7f7f2c154f70 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 -- 172.21.15.49:0/477754043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 msgr2=0x7f7f2c0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 --2- 172.21.15.49:0/477754043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009870 tx=0x7f7f2002a300 comp rx=0 tx=0).stop 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 -- 172.21.15.49:0/477754043 shutdown_connections 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 --2- 172.21.15.49:0/477754043 >> v2:172.21.15.183:3300/0 conn(0x7f7f2c0c6b80 0x7f7f2c071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 --2- 172.21.15.49:0/477754043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c154f70 0x7f7f2c0c6640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 -- 172.21.15.49:0/477754043 >> 172.21.15.49:0/477754043 conn(0x7f7f2c074840 msgr2=0x7f7f2c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 -- 172.21.15.49:0/477754043 shutdown_connections 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.516+0000 7f7f31279640 1 -- 172.21.15.49:0/477754043 wait complete. 2024-05-09T08:56:59.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 Processor -- start 2024-05-09T08:56:59.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 -- start start 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 0x7f7f2c0c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f2c154b90 0x7f7f2c1f1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0c6120 con 0x7f7f2c154f70 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0c5e40 con 0x7f7f2c154b90 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f31279640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c15af40 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f2affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 0x7f7f2c0c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.518+0000 7f7f2b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55716/0 (socket says 172.21.15.49:55716) 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2b7fe640 1 -- 172.21.15.49:0/1839748319 learned_addr learned my addr 172.21.15.49:0/1839748319 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:56:59.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f2c154b90 0x7f7f2c1f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 -- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f2c154b90 msgr2=0x7f7f2c1f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f2c154b90 0x7f7f2c1f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 -- 172.21.15.49:0/1839748319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 msgr2=0x7f7f2c0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 -- 172.21.15.49:0/1839748319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c1f0e40 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2b7fe640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f2affd640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 0x7f7f2c0c8900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c370 tx=0x7f7f1c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0074e0 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c0047e0 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.519+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c153930 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.520+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004bd0 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.520+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c153ef0 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.520+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c04f660 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.521+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f1c01a020 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.522+0000 7f7f137fe640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 0x7f7f1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:56:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.522+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(32..32 src has 1..32) ==== 5857+0+0 (secure 0 0 0) 0x7f7f1c08a640 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.522+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 0x7f7f1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:56:59.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.522+0000 7f7f2a7fc640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 0x7f7f1806e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f20034000 tx=0x7f7f2002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:56:59.523 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:59.522+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:59.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.530+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f1c0587d0 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.573 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.848870052000002, 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:56:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:56:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.848849453, 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:56:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:56:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.848844581999998, 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:56:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.845286283, 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:56:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:56:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:56:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.845278055000001, 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:56:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.845274246000002, 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:56:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.842718218000002, 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:56:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.828217410999997, 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:56:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:56:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:56:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.82820194, 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:56:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.828197195999998, 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:56:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:56:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.828190779000003, 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:56:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:56:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:56:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812214560000001, 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:56:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:56:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812201858000002, 2024-05-09T08:56:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:56:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:56:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:56:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:56:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812193941, 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:56:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:56:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812191432999999, 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:56:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812188969000001, 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:56:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812173674, 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:56:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812169353000002, 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:56:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:56:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:56:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.812166213000001, 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:56:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.809706413000001, 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:56:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:56:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:56:59.695 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.679+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f1c9c15ac80 con 0x7f1c9c154b90 2024-05-09T08:56:59.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:56:59.697 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:59.697+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:59.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:56:59.727+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f7f2c0ccf60 con 0x7f7f2c0c6b80 2024-05-09T08:56:59.797 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:56:59.798+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:56:59.840 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.841 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:59.841 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:56:59.841 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.849 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:56:59.898 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:56:59.899+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:56:59.952 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:56:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:56:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:56:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:56:59.952 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:56:59.960 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:00.001 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:00.002+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:00.100 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:00.101+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:00.103 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.422838507999998, 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.422835640999999, 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.422831995999999, 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.380022830999998, 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.375230842000001, 2024-05-09T08:57:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.372724474000002, 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.371253084000003, 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 42.371177246000002, 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358355836000001, 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358334837999998, 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358330444000003, 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358325385999997, 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358321904999997, 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358317286999998, 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358313817999999, 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.358310517, 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.357997447999999, 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.341921626000001, 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.341916947000001, 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.341910929000001, 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.223 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:00.225 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:00.226 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:00.226+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:00.325 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:00.326+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:00.329 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:00.329 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:00.329 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:00.329 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:00.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:00.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.353+0000 7f1c92ffd640 1 -- 172.21.15.49:0/4028152652 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v33) ==== 91+0+0 (secure 0 0 0) 0x7f1c8405c700 con 0x7f1c9c154b90 2024-05-09T08:57:00.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.353+0000 7f7f137fe640 1 -- 172.21.15.49:0/1839748319 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v33) ==== 113+0+0 (secure 0 0 0) 0x7f7f1c05cad0 con 0x7f7f2c0c6b80 2024-05-09T08:57:00.353 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-05-09T08:57:00.353 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:57:00.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 msgr2=0x7f1c7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 0x7f1c7c06e320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fd0 tx=0x7f1c8c00b040 comp rx=0 tx=0).stop 2024-05-09T08:57:00.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 msgr2=0x7f1c9c14a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 0x7f1c9c14a3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c3c0 tx=0x7f1c84001e70 comp rx=0 tx=0).stop 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 msgr2=0x7f7f1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 0x7f7f1806e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f20034000 tx=0x7f7f2002e000 comp rx=0 tx=0).stop 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 msgr2=0x7f7f2c0c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.356+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 0x7f7f2c0c8900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c370 tx=0x7f7f1c00c830 comp rx=0 tx=0).stop 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 shutdown_connections 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c7c06be60 0x7f1c7c06e320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 shutdown_connections 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f1806be60 0x7f7f1806e320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f2c154f70 0x7f7f2c0c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f2c154b90 0x7f7f2c1f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 --2- 172.21.15.49:0/1839748319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f2c0c6b80 0x7f7f2c0c8900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 >> 172.21.15.49:0/1839748319 conn(0x7f7f2c074840 msgr2=0x7f7f2c0b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c9c154f70 0x7f1c9c14c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c9c154b90 0x7f1c9c14a3f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 --2- 172.21.15.49:0/4028152652 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c9c0c6d50 0x7f1c9c14aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 >> 172.21.15.49:0/4028152652 conn(0x7f1c9c074840 msgr2=0x7f1c9c0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 shutdown_connections 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 shutdown_connections 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f1ca0f14640 1 -- 172.21.15.49:0/4028152652 wait complete. 2024-05-09T08:57:00.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.357+0000 7f7f31279640 1 -- 172.21.15.49:0/1839748319 wait complete. 2024-05-09T08:57:00.370 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-09T08:57:00.426 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:00.427+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:00.481 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:00.481 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:00.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.527+0000 7efeb2d5a640 1 Processor -- start 2024-05-09T08:57:00.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.527+0000 7efeb2d5a640 1 -- start start 2024-05-09T08:57:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb2d5a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efeac154df0 0x7efeac15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb2d5a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 0x7efeac15e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb2d5a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac15c700 con 0x7efeac15e800 2024-05-09T08:57:00.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb2d5a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7efeac071ef0 con 0x7efeac154df0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb2d5a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efeac15ed50 con 0x7efeac1551d0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efeac154df0 0x7efeac15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7efeac154df0 0x7efeac15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55722/0 (socket says 172.21.15.49:55722) 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 -- 172.21.15.49:0/2751449903 learned_addr learned my addr 172.21.15.49:0/2751449903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 -- 172.21.15.49:0/2751449903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 msgr2=0x7efeac15e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeabfff640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 0x7efeac15e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 0x7efeac15e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 -- 172.21.15.49:0/2751449903 >> v1:172.21.15.183:6789/0 conn(0x7efeac15e800 legacy=0x7efeac06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeb0af6640 1 -- 172.21.15.49:0/2751449903 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac15f030 con 0x7efeac154df0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.528+0000 7efeabfff640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 0x7efeac15e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.529+0000 7efeb0af6640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac15c1c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efea4009870 tx=0x7efea402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c95feddc4c8079e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.529+0000 7efeab7fe640 1 -- 172.21.15.49:0/2751449903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4037070 con 0x7efeac154df0 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.529+0000 7efeab7fe640 1 -- 172.21.15.49:0/2751449903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea4032440 con 0x7efeac154df0 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.529+0000 7efeab7fe640 1 -- 172.21.15.49:0/2751449903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4032810 con 0x7efeac154df0 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 -- 172.21.15.49:0/2751449903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 msgr2=0x7efeac15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac15c1c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efea4009870 tx=0x7efea402a2a0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 -- 172.21.15.49:0/2751449903 shutdown_connections 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac1551d0 0x7efeac15e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/2751449903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac15c1c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 -- 172.21.15.49:0/2751449903 >> 172.21.15.49:0/2751449903 conn(0x7efeac06b200 msgr2=0x7efeac06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 -- 172.21.15.49:0/2751449903 shutdown_connections 2024-05-09T08:57:00.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.530+0000 7efeb2d5a640 1 -- 172.21.15.49:0/2751449903 wait complete. 2024-05-09T08:57:00.532 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:00.531+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:00.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.531+0000 7efeb2d5a640 1 Processor -- start 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.531+0000 7efeb2d5a640 1 -- start start 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 0x7efeac1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 0x7efeac14ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7efeac15b480 con 0x7efeac154df0 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efeac15b1a0 con 0x7efeac15e800 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb2d5a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7efeac06f920 con 0x7efeac14bb10 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55726/0 (socket says 172.21.15.49:55726) 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 -- 172.21.15.49:0/1892649094 learned_addr learned my addr 172.21.15.49:0/1892649094 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:00.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb12f7640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 0x7efeac14ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeabfff640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 0x7efeac1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 -- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 msgr2=0x7efeac14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 0x7efeac14ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 -- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 msgr2=0x7efeac1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 0x7efeac1f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 -- 172.21.15.49:0/1892649094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac0ccb60 con 0x7efeac154df0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeabfff640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 0x7efeac1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb12f7640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 0x7efeac14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.532+0000 7efeb0af6640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.533+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4037040 con 0x7efeac154df0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.533+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea4042070 con 0x7efeac154df0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.533+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac14e600 con 0x7efeac154df0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.533+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4030bb0 con 0x7efeac154df0 2024-05-09T08:57:00.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.533+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac0cce00 con 0x7efeac154df0 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.534+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac153460 con 0x7efeac154df0 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.534+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efea4047460 con 0x7efeac154df0 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.535+0000 7efea97fa640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 0x7efe8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.535+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5857+0+0 (secure 0 0 0) 0x7efea40abaf0 con 0x7efeac154df0 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.535+0000 7efeabfff640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 0x7efe8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.535+0000 7efeabfff640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 0x7efe8c06e320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efeac06bb40 tx=0x7efe9c007ab0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.539+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efeac153460 con 0x7efeac154df0 2024-05-09T08:57:00.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:00.628 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:00.629+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:00.637 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:00.637 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:00.638 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.973346401999997, 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:00.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:00.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:00.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580690138000001, 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.647 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:00.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580642546, 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.654 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:00.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:00.660 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580592678999999, 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:00.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580586879000002, 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:00.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:00.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:00.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:00.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580475079000003, 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:00.673 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:00.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:00.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580377917, 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.680 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:00.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:00.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.687 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580372158999999, 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:00.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580298519000003, 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:00.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:00.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580246707000001, 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.700 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:00.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.580242988000002, 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.706 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:00.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.712 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.095186531000003, 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:00.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.718 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.095170047000003, 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:00.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.095088089999997, 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:00.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:00.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.095040529999999, 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:00.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:00.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:00.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.736 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.891412086000003, 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.742 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.875332641, 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:00.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:00.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:00.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:00.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:00.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:00.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.752 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.875328325000002, 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.753 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:00.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:00.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:00.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.875319361999999, 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:00.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:00.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.875315302000001, 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:00.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:00.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:00.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:00.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:00.771 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:00.774 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:00.774+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:00.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.710+0000 7fa7b82bd640 1 Processor -- start 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.710+0000 7fa7b82bd640 1 -- start start 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b82bd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa7b015af60 0x7fa7b00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b82bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 0x7fa7b006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b82bd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b00c4da0 con 0x7fa7b0151be0 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b82bd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b00c5c70 con 0x7fa7b015af60 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b82bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b00c5060 con 0x7fa7b00c71d0 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b5858640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa7b015af60 0x7fa7b00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b6059640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa7b0151be0 0x7fa7b00c45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51252/0 (socket says 172.21.15.49:51252) 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.711+0000 7fa7b6059640 1 -- 172.21.15.49:0/2001404145 learned_addr learned my addr 172.21.15.49:0/2001404145 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:00.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b685a640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 0x7fa7b006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5858640 1 -- 172.21.15.49:0/2001404145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 msgr2=0x7fa7b006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5858640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 0x7fa7b006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5858640 1 -- 172.21.15.49:0/2001404145 >> v1:172.21.15.49:6789/0 conn(0x7fa7b0151be0 legacy=0x7fa7b00c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5858640 1 -- 172.21.15.49:0/2001404145 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b00c5340 con 0x7fa7b015af60 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5057640 1 -- 172.21.15.49:0/2001404145 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597667620 0 0) 0x7fa7b00c4da0 con 0x7fa7b0151be0 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b685a640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 0x7fa7b006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.712+0000 7fa7b5858640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b015af60 0x7fa7b00c6c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0097a0 tx=0x7fa7ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50c5f026a35293f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b5057640 1 -- 172.21.15.49:0/2001404145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac037070 con 0x7fa7b015af60 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b5057640 1 -- 172.21.15.49:0/2001404145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ac032440 con 0x7fa7b015af60 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b5057640 1 -- 172.21.15.49:0/2001404145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac032810 con 0x7fa7b015af60 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 -- 172.21.15.49:0/2001404145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b015af60 msgr2=0x7fa7b00c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b015af60 0x7fa7b00c6c90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0097a0 tx=0x7fa7ac02a210 comp rx=0 tx=0).stop 2024-05-09T08:57:00.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 -- 172.21.15.49:0/2001404145 shutdown_connections 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b00c71d0 0x7fa7b006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/2001404145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b015af60 0x7fa7b00c6c90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 -- 172.21.15.49:0/2001404145 >> 172.21.15.49:0/2001404145 conn(0x7fa7b00cc290 msgr2=0x7fa7b00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 -- 172.21.15.49:0/2001404145 shutdown_connections 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.713+0000 7fa7b82bd640 1 -- 172.21.15.49:0/2001404145 wait complete. 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.714+0000 7fa7b82bd640 1 Processor -- start 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 -- start start 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b0151be0 0x7fa7b0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b015af60 0x7fa7b0158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0151fc0 con 0x7fa7b00c71d0 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b00c5900 con 0x7fa7b015af60 2024-05-09T08:57:00.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b82bd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b0152140 con 0x7fa7b0151be0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b6059640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b6059640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55748/0 (socket says 172.21.15.49:55748) 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b6059640 1 -- 172.21.15.49:0/3202156184 learned_addr learned my addr 172.21.15.49:0/3202156184 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.715+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b0151be0 0x7fa7b0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b015af60 0x7fa7b0158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b0151be0 msgr2=0x7fa7b0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa7b0151be0 0x7fa7b0154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b015af60 msgr2=0x7fa7b0158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7b015af60 0x7fa7b0158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b0155d10 con 0x7fa7b00c71d0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c4d0 tx=0x7fa7a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4018070 con 0x7fa7b00c71d0 2024-05-09T08:57:00.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a4007500 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b01553a0 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4004a20 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.716+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b0152600 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.717+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa7b004f660 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7a401b070 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7a2ffd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5857+0+0 (secure 0 0 0) 0x7fa7a408a250 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v33) ==== 54+0+16 (secure 0 0 0) 0x7fa7a408c8d0 con 0x7fa7b00c71d0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fa7b0159090 0x7fa7b00c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.718+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7b0158820 con 0x7fa7b0159090 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.719+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fa7b0159090 0x7fa7b00c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.783 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.719+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fa7b0159090 0x7fa7b00c7b50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.719+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 0x7fa79406e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac034000 tx=0x7fa7ac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.720+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7a8009b30 con 0x7fa7b0159090 2024-05-09T08:57:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.735+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b0155030 con 0x7fa7b0159090 2024-05-09T08:57:00.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.736+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b0155030 con 0x7fa7b0159090 2024-05-09T08:57:00.784 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.736+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fa7b0144b80 0x7fa7b0144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.736+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa7b01454a0 con 0x7fa7b0144b80 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.737+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fa7b0144b80 0x7fa7b0144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.737+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fa7b0144b80 0x7fa7b0144f60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.737+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b01454a0 con 0x7fa7b0144b80 2024-05-09T08:57:00.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.740+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7efeac04f660 con 0x7efeac154df0 2024-05-09T08:57:00.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.753+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b0061f40 con 0x7fa7b0144b80 2024-05-09T08:57:00.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.754+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b0061f40 con 0x7fa7b0144b80 2024-05-09T08:57:00.786 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:00.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.755+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fa7b00ca160 0x7fa7b00ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.755+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa7b0153a60 con 0x7fa7b00ca160 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.755+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fa7b00ca160 0x7fa7b00ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.755+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fa7b00ca160 0x7fa7b00ca580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.756+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b0153a60 con 0x7fa7b00ca160 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.770+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b00caac0 con 0x7fa7b00ca160 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.771+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b00caac0 con 0x7fa7b00ca160 2024-05-09T08:57:00.788 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.771+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fa7b0152df0 0x7fa7b0153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.772+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa7b00cb030 con 0x7fa7b0152df0 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.772+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fa7b0152df0 0x7fa7b0153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.772+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fa7b0152df0 0x7fa7b0153230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.773+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b00cb030 con 0x7fa7b0152df0 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.787+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b00cb030 con 0x7fa7b0152df0 2024-05-09T08:57:00.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.788+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b00cb030 con 0x7fa7b0152df0 2024-05-09T08:57:00.791 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:00.792 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:00.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.788+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fa7b002c550 0x7fa7b002c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.788+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa7b002ced0 con 0x7fa7b002c550 2024-05-09T08:57:00.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.789+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fa7b002c550 0x7fa7b002c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.792 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.789+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fa7b002c550 0x7fa7b002c990 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.790+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b002ced0 con 0x7fa7b002c550 2024-05-09T08:57:00.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.804+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b002ced0 con 0x7fa7b002c550 2024-05-09T08:57:00.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.806+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b002ced0 con 0x7fa7b002c550 2024-05-09T08:57:00.805 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:00.806 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:00.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.806+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fa7b0202720 0x7fa7b0204b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.806+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa7b02078e0 con 0x7fa7b0202720 2024-05-09T08:57:00.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.806+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fa7b0202720 0x7fa7b0204b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.807+0000 7fa7b5858640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fa7b0202720 0x7fa7b0204b10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.808+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b02078e0 con 0x7fa7b0202720 2024-05-09T08:57:00.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.822+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b0205050 con 0x7fa7b0202720 2024-05-09T08:57:00.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.824+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b0205050 con 0x7fa7b0202720 2024-05-09T08:57:00.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.824+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 0x7fa7b0205bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.824+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa7b020acd0 con 0x7fa7b0205770 2024-05-09T08:57:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.825+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 0x7fa7b0205bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.825+0000 7fa7b685a640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 0x7fa7b0205bb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.826+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b020acd0 con 0x7fa7b0205770 2024-05-09T08:57:00.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.840+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b0206140 con 0x7fa7b0205770 2024-05-09T08:57:00.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.842+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b0206140 con 0x7fa7b0205770 2024-05-09T08:57:00.842 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.842+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 0x7fa7b0206c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:00.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.842+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa7b02072f0 con 0x7fa7b02067c0 2024-05-09T08:57:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.843+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 0x7fa7b0206c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.843+0000 7fa7b6059640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 0x7fa7b0206c20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:00.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.844+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b02072f0 con 0x7fa7b02067c0 2024-05-09T08:57:00.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.858+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa7b0207270 con 0x7fa7b02067c0 2024-05-09T08:57:00.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7a2ffd640 1 -- 172.21.15.49:0/3202156184 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa7b0207270 con 0x7fa7b02067c0 2024-05-09T08:57:00.859 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fa7b0159090 msgr2=0x7fa7b00c7b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fa7b0159090 0x7fa7b00c7b50 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fa7b0144b80 msgr2=0x7fa7b0144f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fa7b0144b80 0x7fa7b0144f60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fa7b00ca160 msgr2=0x7fa7b00ca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fa7b00ca160 0x7fa7b00ca580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fa7b0152df0 msgr2=0x7fa7b0153230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fa7b0152df0 0x7fa7b0153230 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fa7b002c550 msgr2=0x7fa7b002c990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fa7b002c550 0x7fa7b002c990 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fa7b0202720 msgr2=0x7fa7b0204b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fa7b0202720 0x7fa7b0204b10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b685a640 1 -- 172.21.15.49:0/3202156184 reap_dead start 2024-05-09T08:57:00.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 msgr2=0x7fa7b0205bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 0x7fa7b0205bb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b685a640 1 -- 172.21.15.49:0/3202156184 reap_dead start 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b685a640 1 -- 172.21.15.49:0/3202156184 reap_dead start 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b685a640 1 -- 172.21.15.49:0/3202156184 reap_dead start 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 msgr2=0x7fa7b0206c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.860+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 0x7fa7b0206c20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 msgr2=0x7fa79406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 0x7fa79406e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac034000 tx=0x7fa7ac02e000 comp rx=0 tx=0).stop 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 msgr2=0x7fa7b01557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:00.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c4d0 tx=0x7fa7a400c990 comp rx=0 tx=0).stop 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 shutdown_connections 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fa7b02067c0 0x7fa7b0206c20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa7b0205770 0x7fa7b0205bb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa79406be30 0x7fa79406e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 --2- 172.21.15.49:0/3202156184 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa7b00c71d0 0x7fa7b01557d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 >> 172.21.15.49:0/3202156184 conn(0x7fa7b00cc290 msgr2=0x7fa7b006b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:00.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 shutdown_connections 2024-05-09T08:57:00.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:00.861+0000 7fa7b82bd640 1 -- 172.21.15.49:0/3202156184 wait complete. 2024-05-09T08:57:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:00.900 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:00.901+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:00.909 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:00.993 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:00.994+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:01.059 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.059 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:01.109 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:01.110+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:01.176 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:01.178+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.217 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.226 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:01.278 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:01.279+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:01.336 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.336 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.345 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.354+0000 7efea97fa640 1 -- 172.21.15.49:0/1892649094 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v34) ==== 113+0+0 (secure 0 0 0) 0x7efea407a400 con 0x7efeac154df0 2024-05-09T08:57:01.353 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-05-09T08:57:01.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.356+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 msgr2=0x7efe8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.356+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 0x7efe8c06e320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efeac06bb40 tx=0x7efe9c007ab0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 msgr2=0x7efeac0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efea4000c00 tx=0x7efea40046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 shutdown_connections 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7efe8c06be60 0x7efe8c06e320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efeac14bb10 0x7efeac14ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeac15e800 0x7efeac1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 --2- 172.21.15.49:0/1892649094 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7efeac154df0 0x7efeac0cdce0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 >> 172.21.15.49:0/1892649094 conn(0x7efeac06b200 msgr2=0x7efeac071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 shutdown_connections 2024-05-09T08:57:01.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.357+0000 7efeb2d5a640 1 -- 172.21.15.49:0/1892649094 wait complete. 2024-05-09T08:57:01.368 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-09T08:57:01.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:57:01.409 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:01.410+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:01.466 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.467 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:01.479 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:01.480+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:01.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.515+0000 7fb804e8c640 1 Processor -- start 2024-05-09T08:57:01.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.515+0000 7fb804e8c640 1 -- start start 2024-05-09T08:57:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.516+0000 7fb804e8c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb800075200 0x7fb80015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.516+0000 7fb804e8c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8000cc190 0x7fb8000cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.516+0000 7fb804e8c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000ccb20 con 0x7fb8001517f0 2024-05-09T08:57:01.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.516+0000 7fb804e8c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb800151bd0 con 0x7fb8000cc190 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.516+0000 7fb804e8c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb80015e260 con 0x7fb800075200 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fed9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb8000cc190 0x7fb8000cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fe59c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb8001517f0 0x7fb8000ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51256/0 (socket says 172.21.15.49:51256) 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fe59c640 1 -- 172.21.15.49:0/1022984016 learned_addr learned my addr 172.21.15.49:0/1022984016 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fdd9b640 1 --2- 172.21.15.49:0/1022984016 >> v2:172.21.15.183:3300/0 conn(0x7fb800075200 0x7fb80015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fed9d640 1 -- 172.21.15.49:0/1022984016 >> v2:172.21.15.183:3300/0 conn(0x7fb800075200 msgr2=0x7fb80015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fed9d640 1 --2- 172.21.15.49:0/1022984016 >> v2:172.21.15.183:3300/0 conn(0x7fb800075200 0x7fb80015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fed9d640 1 -- 172.21.15.49:0/1022984016 >> v1:172.21.15.49:6789/0 conn(0x7fb8001517f0 legacy=0x7fb8000ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.517+0000 7fb7fed9d640 1 -- 172.21.15.49:0/1022984016 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80015e520 con 0x7fb8000cc190 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb7fed9d640 1 --2- 172.21.15.49:0/1022984016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8000cc5e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8000cdc20 tx=0x7fb7f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a29fbea6d7198 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb7fd59a640 1 -- 172.21.15.49:0/1022984016 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007740 con 0x7fb8000cc190 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb7fd59a640 1 -- 172.21.15.49:0/1022984016 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f80078e0 con 0x7fb8000cc190 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb7fd59a640 1 -- 172.21.15.49:0/1022984016 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007cb0 con 0x7fb8000cc190 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb804e8c640 1 -- 172.21.15.49:0/1022984016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 msgr2=0x7fb8000cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.518+0000 7fb804e8c640 1 --2- 172.21.15.49:0/1022984016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8000cc5e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb8000cdc20 tx=0x7fb7f802a710 comp rx=0 tx=0).stop 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 -- 172.21.15.49:0/1022984016 shutdown_connections 2024-05-09T08:57:01.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 --2- 172.21.15.49:0/1022984016 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8000cc5e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 --2- 172.21.15.49:0/1022984016 >> v2:172.21.15.183:3300/0 conn(0x7fb800075200 0x7fb80015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 -- 172.21.15.49:0/1022984016 >> 172.21.15.49:0/1022984016 conn(0x7fb80006bd00 msgr2=0x7fb800074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 -- 172.21.15.49:0/1022984016 shutdown_connections 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.519+0000 7fb804e8c640 1 -- 172.21.15.49:0/1022984016 wait complete. 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.520+0000 7fb804e8c640 1 Processor -- start 2024-05-09T08:57:01.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.520+0000 7fb804e8c640 1 -- start start 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb800075200 0x7fb8000cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001517f0 0x7fb80014ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb80015edc0 con 0x7fb8000cc190 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80015eae0 con 0x7fb8001517f0 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb80015f0a0 con 0x7fb800075200 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55776/0 (socket says 172.21.15.49:55776) 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 -- 172.21.15.49:0/926799215 learned_addr learned my addr 172.21.15.49:0/926799215 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fed9d640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001517f0 0x7fb80014ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fe59c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb800075200 0x7fb8000cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 -- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb800075200 msgr2=0x7fb8000cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb800075200 0x7fb8000cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 -- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001517f0 msgr2=0x7fb80014ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001517f0 0x7fb80014ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 -- 172.21.15.49:0/926799215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8001f5090 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7fdd9b640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e9b0 tx=0x7fb7e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8017070 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e800c9a0 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001e4bd0 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800cd90 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.521+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001e5190 con 0x7fb8000cc190 2024-05-09T08:57:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.522+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800075850 con 0x7fb8000cc190 2024-05-09T08:57:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.523+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7e801a070 con 0x7fb8000cc190 2024-05-09T08:57:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.523+0000 7fb7f6ffd640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 0x7fb7e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.523+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7fb7e808b250 con 0x7fb8000cc190 2024-05-09T08:57:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.524+0000 7fb7fe59c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 0x7fb7e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.524+0000 7fb7fe59c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 0x7fb7e006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007fd0 tx=0x7fb7f0007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.528+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb800075850 con 0x7fb8000cc190 2024-05-09T08:57:01.580 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:01.581+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:01.588 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.589 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.599 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:57:01.683 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:01.684+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:01.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.712+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8001f5370 con 0x7fb7e006be30 2024-05-09T08:57:01.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.719+0000 7fb7f6ffd640 1 -- 172.21.15.49:0/926799215 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7fb8001f5370 con 0x7fb7e006be30 2024-05-09T08:57:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:57:01.723 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:57:01.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 msgr2=0x7fb7e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 0x7fb7e006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0007fd0 tx=0x7fb7f0007e80 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 msgr2=0x7fb8001f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e9b0 tx=0x7fb7e800ee70 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 shutdown_connections 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb7e006be30 0x7fb7e006e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001517f0 0x7fb80014ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb8000cc190 0x7fb8001f46f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 --2- 172.21.15.49:0/926799215 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb800075200 0x7fb8000cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 >> 172.21.15.49:0/926799215 conn(0x7fb80006bd00 msgr2=0x7fb80014ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 shutdown_connections 2024-05-09T08:57:01.725 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.722+0000 7fb804e8c640 1 -- 172.21.15.49:0/926799215 wait complete. 2024-05-09T08:57:01.733 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-09T08:57:01.358224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601806,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":374,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236952,"kb_used_data":4108,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737768,"statfs":{"total":773094113280,"available":772851474432,"internally_reserved":0,"allocated":4206592,"data_stored":2320503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020565"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349777+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:58.349777+0000","last_peered":"2024-05-09T08:56:58.349777+0000","last_clean":"2024-05-09T08:56:58.349777+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:58.349777+0000","last_undegraded":"2024-05-09T08:56:58.349777+0000","last_fullsized":"2024-05-09T08:56:58.349777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356333+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:57:00.356333+0000","last_peered":"2024-05-09T08:57:00.356333+0000","last_clean":"2024-05-09T08:57:00.356333+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:57:00.356333+0000","last_undegraded":"2024-05-09T08:57:00.356333+0000","last_fullsized":"2024-05-09T08:57:00.356333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355481+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:57:00.355481+0000","last_peered":"2024-05-09T08:57:00.355481+0000","last_clean":"2024-05-09T08:57:00.355481+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:57:00.355481+0000","last_undegraded":"2024-05-09T08:57:00.355481+0000","last_fullsized":"2024-05-09T08:57:00.355481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349443+0000","last_change":"2024-05-09T08:56:50.303447+0000","last_active":"2024-05-09T08:56:58.349443+0000","last_peered":"2024-05-09T08:56:58.349443+0000","last_clean":"2024-05-09T08:56:58.349443+0000","last_became_active":"2024-05-09T08:56:50.303115+0000","last_became_peered":"2024-05-09T08:56:50.303115+0000","last_unstale":"2024-05-09T08:56:58.349443+0000","last_undegraded":"2024-05-09T08:56:58.349443+0000","last_fullsized":"2024-05-09T08:56:58.349443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:18.581413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349659+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:58.349659+0000","last_peered":"2024-05-09T08:56:58.349659+0000","last_clean":"2024-05-09T08:56:58.349659+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:58.349659+0000","last_undegraded":"2024-05-09T08:56:58.349659+0000","last_fullsized":"2024-05-09T08:56:58.349659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349945+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:59.349945+0000","last_peered":"2024-05-09T08:56:59.349945+0000","last_clean":"2024-05-09T08:56:59.349945+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:59.349945+0000","last_undegraded":"2024-05-09T08:56:59.349945+0000","last_fullsized":"2024-05-09T08:56:59.349945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349720+0000","last_change":"2024-05-09T08:56:50.749326+0000","last_active":"2024-05-09T08:56:58.349720+0000","last_peered":"2024-05-09T08:56:58.349720+0000","last_clean":"2024-05-09T08:56:58.349720+0000","last_became_active":"2024-05-09T08:56:50.748996+0000","last_became_peered":"2024-05-09T08:56:50.748996+0000","last_unstale":"2024-05-09T08:56:58.349720+0000","last_undegraded":"2024-05-09T08:56:58.349720+0000","last_fullsized":"2024-05-09T08:56:58.349720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:54.050934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349591+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:59.349591+0000","last_peered":"2024-05-09T08:56:59.349591+0000","last_clean":"2024-05-09T08:56:59.349591+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:59.349591+0000","last_undegraded":"2024-05-09T08:56:59.349591+0000","last_fullsized":"2024-05-09T08:56:59.349591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349826+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:58.349826+0000","last_peered":"2024-05-09T08:56:58.349826+0000","last_clean":"2024-05-09T08:56:58.349826+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:58.349826+0000","last_undegraded":"2024-05-09T08:56:58.349826+0000","last_fullsized":"2024-05-09T08:56:58.349826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350304+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:58.350304+0000","last_peered":"2024-05-09T08:56:58.350304+0000","last_clean":"2024-05-09T08:56:58.350304+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:58.350304+0000","last_undegraded":"2024-05-09T08:56:58.350304+0000","last_fullsized":"2024-05-09T08:56:58.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349522+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:58.349522+0000","last_peered":"2024-05-09T08:56:58.349522+0000","last_clean":"2024-05-09T08:56:58.349522+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:58.349522+0000","last_undegraded":"2024-05-09T08:56:58.349522+0000","last_fullsized":"2024-05-09T08:56:58.349522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354210+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:57:00.354210+0000","last_peered":"2024-05-09T08:57:00.354210+0000","last_clean":"2024-05-09T08:57:00.354210+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:57:00.354210+0000","last_undegraded":"2024-05-09T08:57:00.354210+0000","last_fullsized":"2024-05-09T08:57:00.354210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349977+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:59.349977+0000","last_peered":"2024-05-09T08:56:59.349977+0000","last_clean":"2024-05-09T08:56:59.349977+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:59.349977+0000","last_undegraded":"2024-05-09T08:56:59.349977+0000","last_fullsized":"2024-05-09T08:56:59.349977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355094+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:57:00.355094+0000","last_peered":"2024-05-09T08:57:00.355094+0000","last_clean":"2024-05-09T08:57:00.355094+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:57:00.355094+0000","last_undegraded":"2024-05-09T08:57:00.355094+0000","last_fullsized":"2024-05-09T08:57:00.355094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355020+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:57:00.355020+0000","last_peered":"2024-05-09T08:57:00.355020+0000","last_clean":"2024-05-09T08:57:00.355020+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:57:00.355020+0000","last_undegraded":"2024-05-09T08:57:00.355020+0000","last_fullsized":"2024-05-09T08:57:00.355020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349549+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:59.349549+0000","last_peered":"2024-05-09T08:56:59.349549+0000","last_clean":"2024-05-09T08:56:59.349549+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:59.349549+0000","last_undegraded":"2024-05-09T08:56:59.349549+0000","last_fullsized":"2024-05-09T08:56:59.349549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349841+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:59.349841+0000","last_peered":"2024-05-09T08:56:59.349841+0000","last_clean":"2024-05-09T08:56:59.349841+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:59.349841+0000","last_undegraded":"2024-05-09T08:56:59.349841+0000","last_fullsized":"2024-05-09T08:56:59.349841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349386+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:58.349386+0000","last_peered":"2024-05-09T08:56:58.349386+0000","last_clean":"2024-05-09T08:56:58.349386+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:58.349386+0000","last_undegraded":"2024-05-09T08:56:58.349386+0000","last_fullsized":"2024-05-09T08:56:58.349386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350246+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:58.350246+0000","last_peered":"2024-05-09T08:56:58.350246+0000","last_clean":"2024-05-09T08:56:58.350246+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:58.350246+0000","last_undegraded":"2024-05-09T08:56:58.350246+0000","last_fullsized":"2024-05-09T08:56:58.350246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349499+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:58.349499+0000","last_peered":"2024-05-09T08:56:58.349499+0000","last_clean":"2024-05-09T08:56:58.349499+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:58.349499+0000","last_undegraded":"2024-05-09T08:56:58.349499+0000","last_fullsized":"2024-05-09T08:56:58.349499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355438+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:57:00.355438+0000","last_peered":"2024-05-09T08:57:00.355438+0000","last_clean":"2024-05-09T08:57:00.355438+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:57:00.355438+0000","last_undegraded":"2024-05-09T08:57:00.355438+0000","last_fullsized":"2024-05-09T08:57:00.355438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349987+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:59.349987+0000","last_peered":"2024-05-09T08:56:59.349987+0000","last_clean":"2024-05-09T08:56:59.349987+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:59.349987+0000","last_undegraded":"2024-05-09T08:56:59.349987+0000","last_fullsized":"2024-05-09T08:56:59.349987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349677+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:59.349677+0000","last_peered":"2024-05-09T08:56:59.349677+0000","last_clean":"2024-05-09T08:56:59.349677+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:59.349677+0000","last_undegraded":"2024-05-09T08:56:59.349677+0000","last_fullsized":"2024-05-09T08:56:59.349677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349945+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:59.349945+0000","last_peered":"2024-05-09T08:56:59.349945+0000","last_clean":"2024-05-09T08:56:59.349945+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:59.349945+0000","last_undegraded":"2024-05-09T08:56:59.349945+0000","last_fullsized":"2024-05-09T08:56:59.349945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349892+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:59.349892+0000","last_peered":"2024-05-09T08:56:59.349892+0000","last_clean":"2024-05-09T08:56:59.349892+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:59.349892+0000","last_undegraded":"2024-05-09T08:56:59.349892+0000","last_fullsized":"2024-05-09T08:56:59.349892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350340+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:58.350340+0000","last_peered":"2024-05-09T08:56:58.350340+0000","last_clean":"2024-05-09T08:56:58.350340+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:58.350340+0000","last_undegraded":"2024-05-09T08:56:58.350340+0000","last_fullsized":"2024-05-09T08:56:58.350340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355910+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:57:00.355910+0000","last_peered":"2024-05-09T08:57:00.355910+0000","last_clean":"2024-05-09T08:57:00.355910+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:57:00.355910+0000","last_undegraded":"2024-05-09T08:57:00.355910+0000","last_fullsized":"2024-05-09T08:57:00.355910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354893+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:57:00.354893+0000","last_peered":"2024-05-09T08:57:00.354893+0000","last_clean":"2024-05-09T08:57:00.354893+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:57:00.354893+0000","last_undegraded":"2024-05-09T08:57:00.354893+0000","last_fullsized":"2024-05-09T08:57:00.354893+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355612+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:57:00.355612+0000","last_peered":"2024-05-09T08:57:00.355612+0000","last_clean":"2024-05-09T08:57:00.355612+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:57:00.355612+0000","last_undegraded":"2024-05-09T08:57:00.355612+0000","last_fullsized":"2024-05-09T08:57:00.355612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350092+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:59.350092+0000","last_peered":"2024-05-09T08:56:59.350092+0000","last_clean":"2024-05-09T08:56:59.350092+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:59.350092+0000","last_undegraded":"2024-05-09T08:56:59.350092+0000","last_fullsized":"2024-05-09T08:56:59.350092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349884+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:58.349884+0000","last_peered":"2024-05-09T08:56:58.349884+0000","last_clean":"2024-05-09T08:56:58.349884+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:58.349884+0000","last_undegraded":"2024-05-09T08:56:58.349884+0000","last_fullsized":"2024-05-09T08:56:58.349884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349567+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:58.349567+0000","last_peered":"2024-05-09T08:56:58.349567+0000","last_clean":"2024-05-09T08:56:58.349567+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:58.349567+0000","last_undegraded":"2024-05-09T08:56:58.349567+0000","last_fullsized":"2024-05-09T08:56:58.349567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.353962+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:57:00.353962+0000","last_peered":"2024-05-09T08:57:00.353962+0000","last_clean":"2024-05-09T08:57:00.353962+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:57:00.353962+0000","last_undegraded":"2024-05-09T08:57:00.353962+0000","last_fullsized":"2024-05-09T08:57:00.353962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350125+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:59.350125+0000","last_peered":"2024-05-09T08:56:59.350125+0000","last_clean":"2024-05-09T08:56:59.350125+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:59.350125+0000","last_undegraded":"2024-05-09T08:56:59.350125+0000","last_fullsized":"2024-05-09T08:56:59.350125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356072+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:57:00.356072+0000","last_peered":"2024-05-09T08:57:00.356072+0000","last_clean":"2024-05-09T08:57:00.356072+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:57:00.356072+0000","last_undegraded":"2024-05-09T08:57:00.356072+0000","last_fullsized":"2024-05-09T08:57:00.356072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350079+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:58.350079+0000","last_peered":"2024-05-09T08:56:58.350079+0000","last_clean":"2024-05-09T08:56:58.350079+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:58.350079+0000","last_undegraded":"2024-05-09T08:56:58.350079+0000","last_fullsized":"2024-05-09T08:56:58.350079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349796+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:59.349796+0000","last_peered":"2024-05-09T08:56:59.349796+0000","last_clean":"2024-05-09T08:56:59.349796+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:59.349796+0000","last_undegraded":"2024-05-09T08:56:59.349796+0000","last_fullsized":"2024-05-09T08:56:59.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349661+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:59.349661+0000","last_peered":"2024-05-09T08:56:59.349661+0000","last_clean":"2024-05-09T08:56:59.349661+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:59.349661+0000","last_undegraded":"2024-05-09T08:56:59.349661+0000","last_fullsized":"2024-05-09T08:56:59.349661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350023+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:59.350023+0000","last_peered":"2024-05-09T08:56:59.350023+0000","last_clean":"2024-05-09T08:56:59.350023+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:59.350023+0000","last_undegraded":"2024-05-09T08:56:59.350023+0000","last_fullsized":"2024-05-09T08:56:59.350023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354852+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:57:00.354852+0000","last_peered":"2024-05-09T08:57:00.354852+0000","last_clean":"2024-05-09T08:57:00.354852+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:57:00.354852+0000","last_undegraded":"2024-05-09T08:57:00.354852+0000","last_fullsized":"2024-05-09T08:57:00.354852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349741+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:58.349741+0000","last_peered":"2024-05-09T08:56:58.349741+0000","last_clean":"2024-05-09T08:56:58.349741+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:58.349741+0000","last_undegraded":"2024-05-09T08:56:58.349741+0000","last_fullsized":"2024-05-09T08:56:58.349741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349950+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:59.349950+0000","last_peered":"2024-05-09T08:56:59.349950+0000","last_clean":"2024-05-09T08:56:59.349950+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:59.349950+0000","last_undegraded":"2024-05-09T08:56:59.349950+0000","last_fullsized":"2024-05-09T08:56:59.349950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350040+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:59.350040+0000","last_peered":"2024-05-09T08:56:59.350040+0000","last_clean":"2024-05-09T08:56:59.350040+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:59.350040+0000","last_undegraded":"2024-05-09T08:56:59.350040+0000","last_fullsized":"2024-05-09T08:56:59.350040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355709+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:57:00.355709+0000","last_peered":"2024-05-09T08:57:00.355709+0000","last_clean":"2024-05-09T08:57:00.355709+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:57:00.355709+0000","last_undegraded":"2024-05-09T08:57:00.355709+0000","last_fullsized":"2024-05-09T08:57:00.355709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356074+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:57:00.356074+0000","last_peered":"2024-05-09T08:57:00.356074+0000","last_clean":"2024-05-09T08:57:00.356074+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:57:00.356074+0000","last_undegraded":"2024-05-09T08:57:00.356074+0000","last_fullsized":"2024-05-09T08:57:00.356074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354228+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:57:00.354228+0000","last_peered":"2024-05-09T08:57:00.354228+0000","last_clean":"2024-05-09T08:57:00.354228+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:57:00.354228+0000","last_undegraded":"2024-05-09T08:57:00.354228+0000","last_fullsized":"2024-05-09T08:57:00.354228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349604+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:58.349604+0000","last_peered":"2024-05-09T08:56:58.349604+0000","last_clean":"2024-05-09T08:56:58.349604+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:58.349604+0000","last_undegraded":"2024-05-09T08:56:58.349604+0000","last_fullsized":"2024-05-09T08:56:58.349604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4760,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350129+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:58.350129+0000","last_peered":"2024-05-09T08:56:58.350129+0000","last_clean":"2024-05-09T08:56:58.350129+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:58.350129+0000","last_undegraded":"2024-05-09T08:56:58.350129+0000","last_fullsized":"2024-05-09T08:56:58.350129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355909+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:57:00.355909+0000","last_peered":"2024-05-09T08:57:00.355909+0000","last_clean":"2024-05-09T08:57:00.355909+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:57:00.355909+0000","last_undegraded":"2024-05-09T08:57:00.355909+0000","last_fullsized":"2024-05-09T08:57:00.355909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355105+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:57:00.355105+0000","last_peered":"2024-05-09T08:57:00.355105+0000","last_clean":"2024-05-09T08:57:00.355105+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:57:00.355105+0000","last_undegraded":"2024-05-09T08:57:00.355105+0000","last_fullsized":"2024-05-09T08:57:00.355105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350158+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:59.350158+0000","last_peered":"2024-05-09T08:56:59.350158+0000","last_clean":"2024-05-09T08:56:59.350158+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:59.350158+0000","last_undegraded":"2024-05-09T08:56:59.350158+0000","last_fullsized":"2024-05-09T08:56:59.350158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349654+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:58.349654+0000","last_peered":"2024-05-09T08:56:58.349654+0000","last_clean":"2024-05-09T08:56:58.349654+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:58.349654+0000","last_undegraded":"2024-05-09T08:56:58.349654+0000","last_fullsized":"2024-05-09T08:56:58.349654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350022+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:58.350022+0000","last_peered":"2024-05-09T08:56:58.350022+0000","last_clean":"2024-05-09T08:56:58.350022+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:58.350022+0000","last_undegraded":"2024-05-09T08:56:58.350022+0000","last_fullsized":"2024-05-09T08:56:58.350022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349650+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:58.349650+0000","last_peered":"2024-05-09T08:56:58.349650+0000","last_clean":"2024-05-09T08:56:58.349650+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:58.349650+0000","last_undegraded":"2024-05-09T08:56:58.349650+0000","last_fullsized":"2024-05-09T08:56:58.349650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349239+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:58.349239+0000","last_peered":"2024-05-09T08:56:58.349239+0000","last_clean":"2024-05-09T08:56:58.349239+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:58.349239+0000","last_undegraded":"2024-05-09T08:56:58.349239+0000","last_fullsized":"2024-05-09T08:56:58.349239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349447+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:58.349447+0000","last_peered":"2024-05-09T08:56:58.349447+0000","last_clean":"2024-05-09T08:56:58.349447+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:58.349447+0000","last_undegraded":"2024-05-09T08:56:58.349447+0000","last_fullsized":"2024-05-09T08:56:58.349447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354194+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:57:00.354194+0000","last_peered":"2024-05-09T08:57:00.354194+0000","last_clean":"2024-05-09T08:57:00.354194+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:57:00.354194+0000","last_undegraded":"2024-05-09T08:57:00.354194+0000","last_fullsized":"2024-05-09T08:57:00.354194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349661+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:58.349661+0000","last_peered":"2024-05-09T08:56:58.349661+0000","last_clean":"2024-05-09T08:56:58.349661+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:58.349661+0000","last_undegraded":"2024-05-09T08:56:58.349661+0000","last_fullsized":"2024-05-09T08:56:58.349661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349933+0000","last_change":"2024-05-09T08:56:49.302708+0000","last_active":"2024-05-09T08:56:59.349933+0000","last_peered":"2024-05-09T08:56:59.349933+0000","last_clean":"2024-05-09T08:56:59.349933+0000","last_became_active":"2024-05-09T08:56:49.302353+0000","last_became_peered":"2024-05-09T08:56:49.302353+0000","last_unstale":"2024-05-09T08:56:59.349933+0000","last_undegraded":"2024-05-09T08:56:59.349933+0000","last_fullsized":"2024-05-09T08:56:59.349933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:33.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349885+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:58.349885+0000","last_peered":"2024-05-09T08:56:58.349885+0000","last_clean":"2024-05-09T08:56:58.349885+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:58.349885+0000","last_undegraded":"2024-05-09T08:56:58.349885+0000","last_fullsized":"2024-05-09T08:56:58.349885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356249+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:57:00.356249+0000","last_peered":"2024-05-09T08:57:00.356249+0000","last_clean":"2024-05-09T08:57:00.356249+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:57:00.356249+0000","last_undegraded":"2024-05-09T08:57:00.356249+0000","last_fullsized":"2024-05-09T08:57:00.356249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349720+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:59.349720+0000","last_peered":"2024-05-09T08:56:59.349720+0000","last_clean":"2024-05-09T08:56:59.349720+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:59.349720+0000","last_undegraded":"2024-05-09T08:56:59.349720+0000","last_fullsized":"2024-05-09T08:56:59.349720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349603+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:58.349603+0000","last_peered":"2024-05-09T08:56:58.349603+0000","last_clean":"2024-05-09T08:56:58.349603+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:58.349603+0000","last_undegraded":"2024-05-09T08:56:58.349603+0000","last_fullsized":"2024-05-09T08:56:58.349603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350159+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:58.350159+0000","last_peered":"2024-05-09T08:56:58.350159+0000","last_clean":"2024-05-09T08:56:58.350159+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:58.350159+0000","last_undegraded":"2024-05-09T08:56:58.350159+0000","last_fullsized":"2024-05-09T08:56:58.350159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349406+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:58.349406+0000","last_peered":"2024-05-09T08:56:58.349406+0000","last_clean":"2024-05-09T08:56:58.349406+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:58.349406+0000","last_undegraded":"2024-05-09T08:56:58.349406+0000","last_fullsized":"2024-05-09T08:56:58.349406+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349597+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:59.349597+0000","last_peered":"2024-05-09T08:56:59.349597+0000","last_clean":"2024-05-09T08:56:59.349597+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:59.349597+0000","last_undegraded":"2024-05-09T08:56:59.349597+0000","last_fullsized":"2024-05-09T08:56:59.349597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349914+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:59.349914+0000","last_peered":"2024-05-09T08:56:59.349914+0000","last_clean":"2024-05-09T08:56:59.349914+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:59.349914+0000","last_undegraded":"2024-05-09T08:56:59.349914+0000","last_fullsized":"2024-05-09T08:56:59.349914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349296+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:58.349296+0000","last_peered":"2024-05-09T08:56:58.349296+0000","last_clean":"2024-05-09T08:56:58.349296+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:58.349296+0000","last_undegraded":"2024-05-09T08:56:58.349296+0000","last_fullsized":"2024-05-09T08:56:58.349296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355728+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:57:00.355728+0000","last_peered":"2024-05-09T08:57:00.355728+0000","last_clean":"2024-05-09T08:57:00.355728+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:57:00.355728+0000","last_undegraded":"2024-05-09T08:57:00.355728+0000","last_fullsized":"2024-05-09T08:57:00.355728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349995+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:58.349995+0000","last_peered":"2024-05-09T08:56:58.349995+0000","last_clean":"2024-05-09T08:56:58.349995+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:58.349995+0000","last_undegraded":"2024-05-09T08:56:58.349995+0000","last_fullsized":"2024-05-09T08:56:58.349995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355327+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:57:00.355327+0000","last_peered":"2024-05-09T08:57:00.355327+0000","last_clean":"2024-05-09T08:57:00.355327+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:57:00.355327+0000","last_undegraded":"2024-05-09T08:57:00.355327+0000","last_fullsized":"2024-05-09T08:57:00.355327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350216+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:59.350216+0000","last_peered":"2024-05-09T08:56:59.350216+0000","last_clean":"2024-05-09T08:56:59.350216+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:59.350216+0000","last_undegraded":"2024-05-09T08:56:59.350216+0000","last_fullsized":"2024-05-09T08:56:59.350216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354167+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:57:00.354167+0000","last_peered":"2024-05-09T08:57:00.354167+0000","last_clean":"2024-05-09T08:57:00.354167+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:57:00.354167+0000","last_undegraded":"2024-05-09T08:57:00.354167+0000","last_fullsized":"2024-05-09T08:57:00.354167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356231+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:57:00.356231+0000","last_peered":"2024-05-09T08:57:00.356231+0000","last_clean":"2024-05-09T08:57:00.356231+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:57:00.356231+0000","last_undegraded":"2024-05-09T08:57:00.356231+0000","last_fullsized":"2024-05-09T08:57:00.356231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350132+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:59.350132+0000","last_peered":"2024-05-09T08:56:59.350132+0000","last_clean":"2024-05-09T08:56:59.350132+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:59.350132+0000","last_undegraded":"2024-05-09T08:56:59.350132+0000","last_fullsized":"2024-05-09T08:56:59.350132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350337+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:58.350337+0000","last_peered":"2024-05-09T08:56:58.350337+0000","last_clean":"2024-05-09T08:56:58.350337+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:58.350337+0000","last_undegraded":"2024-05-09T08:56:58.350337+0000","last_fullsized":"2024-05-09T08:56:58.350337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355083+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:57:00.355083+0000","last_peered":"2024-05-09T08:57:00.355083+0000","last_clean":"2024-05-09T08:57:00.355083+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:57:00.355083+0000","last_undegraded":"2024-05-09T08:57:00.355083+0000","last_fullsized":"2024-05-09T08:57:00.355083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'8","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356385+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:57:00.356385+0000","last_peered":"2024-05-09T08:57:00.356385+0000","last_clean":"2024-05-09T08:57:00.356385+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:57:00.356385+0000","last_undegraded":"2024-05-09T08:57:00.356385+0000","last_fullsized":"2024-05-09T08:57:00.356385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350358+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:59.350358+0000","last_peered":"2024-05-09T08:56:59.350358+0000","last_clean":"2024-05-09T08:56:59.350358+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:59.350358+0000","last_undegraded":"2024-05-09T08:56:59.350358+0000","last_fullsized":"2024-05-09T08:56:59.350358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349848+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:58.349848+0000","last_peered":"2024-05-09T08:56:58.349848+0000","last_clean":"2024-05-09T08:56:58.349848+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:58.349848+0000","last_undegraded":"2024-05-09T08:56:58.349848+0000","last_fullsized":"2024-05-09T08:56:58.349848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355915+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:57:00.355915+0000","last_peered":"2024-05-09T08:57:00.355915+0000","last_clean":"2024-05-09T08:57:00.355915+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:57:00.355915+0000","last_undegraded":"2024-05-09T08:57:00.355915+0000","last_fullsized":"2024-05-09T08:57:00.355915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349171+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:58.349171+0000","last_peered":"2024-05-09T08:56:58.349171+0000","last_clean":"2024-05-09T08:56:58.349171+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:58.349171+0000","last_undegraded":"2024-05-09T08:56:58.349171+0000","last_fullsized":"2024-05-09T08:56:58.349171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354044+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:57:00.354044+0000","last_peered":"2024-05-09T08:57:00.354044+0000","last_clean":"2024-05-09T08:57:00.354044+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:57:00.354044+0000","last_undegraded":"2024-05-09T08:57:00.354044+0000","last_fullsized":"2024-05-09T08:57:00.354044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355180+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:57:00.355180+0000","last_peered":"2024-05-09T08:57:00.355180+0000","last_clean":"2024-05-09T08:57:00.355180+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:57:00.355180+0000","last_undegraded":"2024-05-09T08:57:00.355180+0000","last_fullsized":"2024-05-09T08:57:00.355180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349749+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:59.349749+0000","last_peered":"2024-05-09T08:56:59.349749+0000","last_clean":"2024-05-09T08:56:59.349749+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:59.349749+0000","last_undegraded":"2024-05-09T08:56:59.349749+0000","last_fullsized":"2024-05-09T08:56:59.349749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355638+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:57:00.355638+0000","last_peered":"2024-05-09T08:57:00.355638+0000","last_clean":"2024-05-09T08:57:00.355638+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:57:00.355638+0000","last_undegraded":"2024-05-09T08:57:00.355638+0000","last_fullsized":"2024-05-09T08:57:00.355638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350075+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:59.350075+0000","last_peered":"2024-05-09T08:56:59.350075+0000","last_clean":"2024-05-09T08:56:59.350075+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:59.350075+0000","last_undegraded":"2024-05-09T08:56:59.350075+0000","last_fullsized":"2024-05-09T08:56:59.350075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349787+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:58.349787+0000","last_peered":"2024-05-09T08:56:58.349787+0000","last_clean":"2024-05-09T08:56:58.349787+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:58.349787+0000","last_undegraded":"2024-05-09T08:56:58.349787+0000","last_fullsized":"2024-05-09T08:56:58.349787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350021+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:58.350021+0000","last_peered":"2024-05-09T08:56:58.350021+0000","last_clean":"2024-05-09T08:56:58.350021+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:58.350021+0000","last_undegraded":"2024-05-09T08:56:58.350021+0000","last_fullsized":"2024-05-09T08:56:58.350021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349575+0000","last_change":"2024-05-09T08:56:50.303865+0000","last_active":"2024-05-09T08:56:58.349575+0000","last_peered":"2024-05-09T08:56:58.349575+0000","last_clean":"2024-05-09T08:56:58.349575+0000","last_became_active":"2024-05-09T08:56:50.303517+0000","last_became_peered":"2024-05-09T08:56:50.303517+0000","last_unstale":"2024-05-09T08:56:58.349575+0000","last_undegraded":"2024-05-09T08:56:58.349575+0000","last_fullsized":"2024-05-09T08:56:58.349575+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:17:17.229833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349425+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:58.349425+0000","last_peered":"2024-05-09T08:56:58.349425+0000","last_clean":"2024-05-09T08:56:58.349425+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:58.349425+0000","last_undegraded":"2024-05-09T08:56:58.349425+0000","last_fullsized":"2024-05-09T08:56:58.349425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349631+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:59.349631+0000","last_peered":"2024-05-09T08:56:59.349631+0000","last_clean":"2024-05-09T08:56:59.349631+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:59.349631+0000","last_undegraded":"2024-05-09T08:56:59.349631+0000","last_fullsized":"2024-05-09T08:56:59.349631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350248+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:58.350248+0000","last_peered":"2024-05-09T08:56:58.350248+0000","last_clean":"2024-05-09T08:56:58.350248+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:58.350248+0000","last_undegraded":"2024-05-09T08:56:58.350248+0000","last_fullsized":"2024-05-09T08:56:58.350248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355604+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:57:00.355604+0000","last_peered":"2024-05-09T08:57:00.355604+0000","last_clean":"2024-05-09T08:57:00.355604+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:57:00.355604+0000","last_undegraded":"2024-05-09T08:57:00.355604+0000","last_fullsized":"2024-05-09T08:57:00.355604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350056+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:59.350056+0000","last_peered":"2024-05-09T08:56:59.350056+0000","last_clean":"2024-05-09T08:56:59.350056+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:59.350056+0000","last_undegraded":"2024-05-09T08:56:59.350056+0000","last_fullsized":"2024-05-09T08:56:59.350056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349821+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:59.349821+0000","last_peered":"2024-05-09T08:56:59.349821+0000","last_clean":"2024-05-09T08:56:59.349821+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:59.349821+0000","last_undegraded":"2024-05-09T08:56:59.349821+0000","last_fullsized":"2024-05-09T08:56:59.349821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356003+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:57:00.356003+0000","last_peered":"2024-05-09T08:57:00.356003+0000","last_clean":"2024-05-09T08:57:00.356003+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:57:00.356003+0000","last_undegraded":"2024-05-09T08:57:00.356003+0000","last_fullsized":"2024-05-09T08:57:00.356003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11438,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":215,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":730061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":142887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":147334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":144732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":385024,"data_stored":149242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":372736,"data_stored":138799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":138869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":728579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":5550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:57:01.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.752 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:57:01.782 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:01.783+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:01.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.857+0000 7f3541ca5640 1 Processor -- start 2024-05-09T08:57:01.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.857+0000 7f3541ca5640 1 -- start start 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f3541ca5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f353c1517f0 0x7f353c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f3541ca5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f3541ca5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f353c15e260 con 0x7f353c0cd520 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f3541ca5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f353c151bd0 con 0x7f353c1517f0 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f3541ca5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f353c15e520 con 0x7f353c075200 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:35110/0 (socket says 172.21.15.49:35110) 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 -- 172.21.15.49:0/1338516125 learned_addr learned my addr 172.21.15.49:0/1338516125 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353b7fe640 1 --2- 172.21.15.49:0/1338516125 >> v2:172.21.15.183:3300/0 conn(0x7f353c1517f0 0x7f353c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 -- 172.21.15.49:0/1338516125 >> v2:172.21.15.183:3300/0 conn(0x7f353c1517f0 msgr2=0x7f353c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:01.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 --2- 172.21.15.49:0/1338516125 >> v2:172.21.15.183:3300/0 conn(0x7f353c1517f0 0x7f353c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 -- 172.21.15.49:0/1338516125 >> v1:172.21.15.183:6789/0 conn(0x7f353c0cd520 legacy=0x7f353c0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.858+0000 7f353affd640 1 -- 172.21.15.49:0/1338516125 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c15e800 con 0x7f353c075200 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.859+0000 7f353affd640 1 --2- 172.21.15.49:0/1338516125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098a0 tx=0x7f352c02a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26a697baebcd3800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.859+0000 7f353a7fc640 1 -- 172.21.15.49:0/1338516125 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c037070 con 0x7f353c075200 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.859+0000 7f353a7fc640 1 -- 172.21.15.49:0/1338516125 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c032640 con 0x7f353c075200 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.859+0000 7f353a7fc640 1 -- 172.21.15.49:0/1338516125 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c032a10 con 0x7f353c075200 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 -- 172.21.15.49:0/1338516125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 msgr2=0x7f353c0ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 --2- 172.21.15.49:0/1338516125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098a0 tx=0x7f352c02a320 comp rx=0 tx=0).stop 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 -- 172.21.15.49:0/1338516125 shutdown_connections 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 --2- 172.21.15.49:0/1338516125 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0ccfe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 --2- 172.21.15.49:0/1338516125 >> v2:172.21.15.183:3300/0 conn(0x7f353c1517f0 0x7f353c15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 -- 172.21.15.49:0/1338516125 >> 172.21.15.49:0/1338516125 conn(0x7f353c06bd00 msgr2=0x7f353c074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:01.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 -- 172.21.15.49:0/1338516125 shutdown_connections 2024-05-09T08:57:01.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.860+0000 7f3541ca5640 1 -- 172.21.15.49:0/1338516125 wait complete. 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.861+0000 7f3541ca5640 1 Processor -- start 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.861+0000 7f3541ca5640 1 -- start start 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353c1517f0 0x7f353c1e6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f353c15f0a0 con 0x7f353c0cd520 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f353c15edc0 con 0x7f353c1517f0 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f3541ca5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f353c0cbc70 con 0x7f353c075200 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f353affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f353affd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55782/0 (socket says 172.21.15.49:55782) 2024-05-09T08:57:01.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f353affd640 1 -- 172.21.15.49:0/2872944750 learned_addr learned my addr 172.21.15.49:0/2872944750 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f353b7fe640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.862+0000 7f353bfff640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353c1517f0 0x7f353c1e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 -- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 msgr2=0x7f353c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 -- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353c1517f0 msgr2=0x7f353c1e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353c1517f0 0x7f353c1e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 -- 172.21.15.49:0/2872944750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c1e58c0 con 0x7f353c0cd520 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f353affd640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f352c000c00 tx=0x7f352c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c037040 con 0x7f353c0cd520 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c042070 con 0x7f353c0cd520 2024-05-09T08:57:01.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353c0b5170 con 0x7f353c0cd520 2024-05-09T08:57:01.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0309b0 con 0x7f353c0cd520 2024-05-09T08:57:01.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.863+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353c1e5020 con 0x7f353c0cd520 2024-05-09T08:57:01.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.864+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353c068560 con 0x7f353c0cd520 2024-05-09T08:57:01.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.865+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f352c030b50 con 0x7f353c0cd520 2024-05-09T08:57:01.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.865+0000 7f3533fff640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 0x7f351c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:01.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.865+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f352c0ab700 con 0x7f353c0cd520 2024-05-09T08:57:01.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.865+0000 7f353b7fe640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 0x7f351c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:01.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.866+0000 7f353b7fe640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 0x7f351c06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ad0 tx=0x7f3524008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:01.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:01.869+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f353c1e58c0 con 0x7f353c0cd520 2024-05-09T08:57:01.883 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:01.884+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:01.913 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:01.913 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:01.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:57:01.984 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:01.985+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:02.032 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.033 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:02.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.051+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f353c0caa00 con 0x7f351c06be30 2024-05-09T08:57:02.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.057+0000 7f3533fff640 1 -- 172.21.15.49:0/2872944750 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7f353c0caa00 con 0x7f351c06be30 2024-05-09T08:57:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:57:02.060 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:57:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 msgr2=0x7f351c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 0x7f351c06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3524009ad0 tx=0x7f3524008040 comp rx=0 tx=0).stop 2024-05-09T08:57:02.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 msgr2=0x7f353c1e61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f352c000c00 tx=0x7f352c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 shutdown_connections 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f351c06be30 0x7f351c06e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f353c1517f0 0x7f353c1e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f353c0cd520 0x7f353c1e61f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 --2- 172.21.15.49:0/2872944750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f353c075200 0x7f353c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 >> 172.21.15.49:0/2872944750 conn(0x7f353c06bd00 msgr2=0x7f353c1504c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 shutdown_connections 2024-05-09T08:57:02.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.060+0000 7f3541ca5640 1 -- 172.21.15.49:0/2872944750 wait complete. 2024-05-09T08:57:02.070 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-05-09T08:57:01.358224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601806,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":374,"num_write_kb":2488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236952,"kb_used_data":4108,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737768,"statfs":{"total":773094113280,"available":772851474432,"internally_reserved":0,"allocated":4206592,"data_stored":2320503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020565"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349777+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:56:58.349777+0000","last_peered":"2024-05-09T08:56:58.349777+0000","last_clean":"2024-05-09T08:56:58.349777+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:56:58.349777+0000","last_undegraded":"2024-05-09T08:56:58.349777+0000","last_fullsized":"2024-05-09T08:56:58.349777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356333+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:57:00.356333+0000","last_peered":"2024-05-09T08:57:00.356333+0000","last_clean":"2024-05-09T08:57:00.356333+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:57:00.356333+0000","last_undegraded":"2024-05-09T08:57:00.356333+0000","last_fullsized":"2024-05-09T08:57:00.356333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355481+0000","last_change":"2024-05-09T08:56:01.686783+0000","last_active":"2024-05-09T08:57:00.355481+0000","last_peered":"2024-05-09T08:57:00.355481+0000","last_clean":"2024-05-09T08:57:00.355481+0000","last_became_active":"2024-05-09T08:56:01.686487+0000","last_became_peered":"2024-05-09T08:56:01.686487+0000","last_unstale":"2024-05-09T08:57:00.355481+0000","last_undegraded":"2024-05-09T08:57:00.355481+0000","last_fullsized":"2024-05-09T08:57:00.355481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:35.621383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349443+0000","last_change":"2024-05-09T08:56:50.303447+0000","last_active":"2024-05-09T08:56:58.349443+0000","last_peered":"2024-05-09T08:56:58.349443+0000","last_clean":"2024-05-09T08:56:58.349443+0000","last_became_active":"2024-05-09T08:56:50.303115+0000","last_became_peered":"2024-05-09T08:56:50.303115+0000","last_unstale":"2024-05-09T08:56:58.349443+0000","last_undegraded":"2024-05-09T08:56:58.349443+0000","last_fullsized":"2024-05-09T08:56:58.349443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:18.581413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349659+0000","last_change":"2024-05-09T08:56:01.695049+0000","last_active":"2024-05-09T08:56:58.349659+0000","last_peered":"2024-05-09T08:56:58.349659+0000","last_clean":"2024-05-09T08:56:58.349659+0000","last_became_active":"2024-05-09T08:56:01.694690+0000","last_became_peered":"2024-05-09T08:56:01.694690+0000","last_unstale":"2024-05-09T08:56:58.349659+0000","last_undegraded":"2024-05-09T08:56:58.349659+0000","last_fullsized":"2024-05-09T08:56:58.349659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:47:03.812012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349945+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:56:59.349945+0000","last_peered":"2024-05-09T08:56:59.349945+0000","last_clean":"2024-05-09T08:56:59.349945+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:56:59.349945+0000","last_undegraded":"2024-05-09T08:56:59.349945+0000","last_fullsized":"2024-05-09T08:56:59.349945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349720+0000","last_change":"2024-05-09T08:56:50.749326+0000","last_active":"2024-05-09T08:56:58.349720+0000","last_peered":"2024-05-09T08:56:58.349720+0000","last_clean":"2024-05-09T08:56:58.349720+0000","last_became_active":"2024-05-09T08:56:50.748996+0000","last_became_peered":"2024-05-09T08:56:50.748996+0000","last_unstale":"2024-05-09T08:56:58.349720+0000","last_undegraded":"2024-05-09T08:56:58.349720+0000","last_fullsized":"2024-05-09T08:56:58.349720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:11:54.050934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349591+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:56:59.349591+0000","last_peered":"2024-05-09T08:56:59.349591+0000","last_clean":"2024-05-09T08:56:59.349591+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:56:59.349591+0000","last_undegraded":"2024-05-09T08:56:59.349591+0000","last_fullsized":"2024-05-09T08:56:59.349591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349826+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:56:58.349826+0000","last_peered":"2024-05-09T08:56:58.349826+0000","last_clean":"2024-05-09T08:56:58.349826+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:56:58.349826+0000","last_undegraded":"2024-05-09T08:56:58.349826+0000","last_fullsized":"2024-05-09T08:56:58.349826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350304+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:56:58.350304+0000","last_peered":"2024-05-09T08:56:58.350304+0000","last_clean":"2024-05-09T08:56:58.350304+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:56:58.350304+0000","last_undegraded":"2024-05-09T08:56:58.350304+0000","last_fullsized":"2024-05-09T08:56:58.350304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349522+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:56:58.349522+0000","last_peered":"2024-05-09T08:56:58.349522+0000","last_clean":"2024-05-09T08:56:58.349522+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:56:58.349522+0000","last_undegraded":"2024-05-09T08:56:58.349522+0000","last_fullsized":"2024-05-09T08:56:58.349522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354210+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:57:00.354210+0000","last_peered":"2024-05-09T08:57:00.354210+0000","last_clean":"2024-05-09T08:57:00.354210+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:57:00.354210+0000","last_undegraded":"2024-05-09T08:57:00.354210+0000","last_fullsized":"2024-05-09T08:57:00.354210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349977+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:56:59.349977+0000","last_peered":"2024-05-09T08:56:59.349977+0000","last_clean":"2024-05-09T08:56:59.349977+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:56:59.349977+0000","last_undegraded":"2024-05-09T08:56:59.349977+0000","last_fullsized":"2024-05-09T08:56:59.349977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355094+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:57:00.355094+0000","last_peered":"2024-05-09T08:57:00.355094+0000","last_clean":"2024-05-09T08:57:00.355094+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:57:00.355094+0000","last_undegraded":"2024-05-09T08:57:00.355094+0000","last_fullsized":"2024-05-09T08:57:00.355094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355020+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:57:00.355020+0000","last_peered":"2024-05-09T08:57:00.355020+0000","last_clean":"2024-05-09T08:57:00.355020+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:57:00.355020+0000","last_undegraded":"2024-05-09T08:57:00.355020+0000","last_fullsized":"2024-05-09T08:57:00.355020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349549+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:56:59.349549+0000","last_peered":"2024-05-09T08:56:59.349549+0000","last_clean":"2024-05-09T08:56:59.349549+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:56:59.349549+0000","last_undegraded":"2024-05-09T08:56:59.349549+0000","last_fullsized":"2024-05-09T08:56:59.349549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349841+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:56:59.349841+0000","last_peered":"2024-05-09T08:56:59.349841+0000","last_clean":"2024-05-09T08:56:59.349841+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:56:59.349841+0000","last_undegraded":"2024-05-09T08:56:59.349841+0000","last_fullsized":"2024-05-09T08:56:59.349841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349386+0000","last_change":"2024-05-09T08:56:01.698885+0000","last_active":"2024-05-09T08:56:58.349386+0000","last_peered":"2024-05-09T08:56:58.349386+0000","last_clean":"2024-05-09T08:56:58.349386+0000","last_became_active":"2024-05-09T08:56:01.698553+0000","last_became_peered":"2024-05-09T08:56:01.698553+0000","last_unstale":"2024-05-09T08:56:58.349386+0000","last_undegraded":"2024-05-09T08:56:58.349386+0000","last_fullsized":"2024-05-09T08:56:58.349386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:26:54.831318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350246+0000","last_change":"2024-05-09T08:56:01.696941+0000","last_active":"2024-05-09T08:56:58.350246+0000","last_peered":"2024-05-09T08:56:58.350246+0000","last_clean":"2024-05-09T08:56:58.350246+0000","last_became_active":"2024-05-09T08:56:01.696664+0000","last_became_peered":"2024-05-09T08:56:01.696664+0000","last_unstale":"2024-05-09T08:56:58.350246+0000","last_undegraded":"2024-05-09T08:56:58.350246+0000","last_fullsized":"2024-05-09T08:56:58.350246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:35:47.778916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349499+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:56:58.349499+0000","last_peered":"2024-05-09T08:56:58.349499+0000","last_clean":"2024-05-09T08:56:58.349499+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:56:58.349499+0000","last_undegraded":"2024-05-09T08:56:58.349499+0000","last_fullsized":"2024-05-09T08:56:58.349499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355438+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:57:00.355438+0000","last_peered":"2024-05-09T08:57:00.355438+0000","last_clean":"2024-05-09T08:57:00.355438+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:57:00.355438+0000","last_undegraded":"2024-05-09T08:57:00.355438+0000","last_fullsized":"2024-05-09T08:57:00.355438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349987+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:56:59.349987+0000","last_peered":"2024-05-09T08:56:59.349987+0000","last_clean":"2024-05-09T08:56:59.349987+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:56:59.349987+0000","last_undegraded":"2024-05-09T08:56:59.349987+0000","last_fullsized":"2024-05-09T08:56:59.349987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349677+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:56:59.349677+0000","last_peered":"2024-05-09T08:56:59.349677+0000","last_clean":"2024-05-09T08:56:59.349677+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:56:59.349677+0000","last_undegraded":"2024-05-09T08:56:59.349677+0000","last_fullsized":"2024-05-09T08:56:59.349677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349945+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:56:59.349945+0000","last_peered":"2024-05-09T08:56:59.349945+0000","last_clean":"2024-05-09T08:56:59.349945+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:56:59.349945+0000","last_undegraded":"2024-05-09T08:56:59.349945+0000","last_fullsized":"2024-05-09T08:56:59.349945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349892+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:56:59.349892+0000","last_peered":"2024-05-09T08:56:59.349892+0000","last_clean":"2024-05-09T08:56:59.349892+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:56:59.349892+0000","last_undegraded":"2024-05-09T08:56:59.349892+0000","last_fullsized":"2024-05-09T08:56:59.349892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350340+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:56:58.350340+0000","last_peered":"2024-05-09T08:56:58.350340+0000","last_clean":"2024-05-09T08:56:58.350340+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:56:58.350340+0000","last_undegraded":"2024-05-09T08:56:58.350340+0000","last_fullsized":"2024-05-09T08:56:58.350340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355910+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:57:00.355910+0000","last_peered":"2024-05-09T08:57:00.355910+0000","last_clean":"2024-05-09T08:57:00.355910+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:57:00.355910+0000","last_undegraded":"2024-05-09T08:57:00.355910+0000","last_fullsized":"2024-05-09T08:57:00.355910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354893+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:57:00.354893+0000","last_peered":"2024-05-09T08:57:00.354893+0000","last_clean":"2024-05-09T08:57:00.354893+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:57:00.354893+0000","last_undegraded":"2024-05-09T08:57:00.354893+0000","last_fullsized":"2024-05-09T08:57:00.354893+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355612+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:57:00.355612+0000","last_peered":"2024-05-09T08:57:00.355612+0000","last_clean":"2024-05-09T08:57:00.355612+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:57:00.355612+0000","last_undegraded":"2024-05-09T08:57:00.355612+0000","last_fullsized":"2024-05-09T08:57:00.355612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350092+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:56:59.350092+0000","last_peered":"2024-05-09T08:56:59.350092+0000","last_clean":"2024-05-09T08:56:59.350092+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:56:59.350092+0000","last_undegraded":"2024-05-09T08:56:59.350092+0000","last_fullsized":"2024-05-09T08:56:59.350092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349884+0000","last_change":"2024-05-09T08:56:01.688495+0000","last_active":"2024-05-09T08:56:58.349884+0000","last_peered":"2024-05-09T08:56:58.349884+0000","last_clean":"2024-05-09T08:56:58.349884+0000","last_became_active":"2024-05-09T08:56:01.688202+0000","last_became_peered":"2024-05-09T08:56:01.688202+0000","last_unstale":"2024-05-09T08:56:58.349884+0000","last_undegraded":"2024-05-09T08:56:58.349884+0000","last_fullsized":"2024-05-09T08:56:58.349884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:31:43.740410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349567+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:56:58.349567+0000","last_peered":"2024-05-09T08:56:58.349567+0000","last_clean":"2024-05-09T08:56:58.349567+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:56:58.349567+0000","last_undegraded":"2024-05-09T08:56:58.349567+0000","last_fullsized":"2024-05-09T08:56:58.349567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.353962+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:57:00.353962+0000","last_peered":"2024-05-09T08:57:00.353962+0000","last_clean":"2024-05-09T08:57:00.353962+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:57:00.353962+0000","last_undegraded":"2024-05-09T08:57:00.353962+0000","last_fullsized":"2024-05-09T08:57:00.353962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350125+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:56:59.350125+0000","last_peered":"2024-05-09T08:56:59.350125+0000","last_clean":"2024-05-09T08:56:59.350125+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:56:59.350125+0000","last_undegraded":"2024-05-09T08:56:59.350125+0000","last_fullsized":"2024-05-09T08:56:59.350125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356072+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:57:00.356072+0000","last_peered":"2024-05-09T08:57:00.356072+0000","last_clean":"2024-05-09T08:57:00.356072+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:57:00.356072+0000","last_undegraded":"2024-05-09T08:57:00.356072+0000","last_fullsized":"2024-05-09T08:57:00.356072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350079+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:56:58.350079+0000","last_peered":"2024-05-09T08:56:58.350079+0000","last_clean":"2024-05-09T08:56:58.350079+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:56:58.350079+0000","last_undegraded":"2024-05-09T08:56:58.350079+0000","last_fullsized":"2024-05-09T08:56:58.350079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349796+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:56:59.349796+0000","last_peered":"2024-05-09T08:56:59.349796+0000","last_clean":"2024-05-09T08:56:59.349796+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:56:59.349796+0000","last_undegraded":"2024-05-09T08:56:59.349796+0000","last_fullsized":"2024-05-09T08:56:59.349796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349661+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:56:59.349661+0000","last_peered":"2024-05-09T08:56:59.349661+0000","last_clean":"2024-05-09T08:56:59.349661+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:56:59.349661+0000","last_undegraded":"2024-05-09T08:56:59.349661+0000","last_fullsized":"2024-05-09T08:56:59.349661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":40,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350023+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:56:59.350023+0000","last_peered":"2024-05-09T08:56:59.350023+0000","last_clean":"2024-05-09T08:56:59.350023+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:56:59.350023+0000","last_undegraded":"2024-05-09T08:56:59.350023+0000","last_fullsized":"2024-05-09T08:56:59.350023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354852+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:57:00.354852+0000","last_peered":"2024-05-09T08:57:00.354852+0000","last_clean":"2024-05-09T08:57:00.354852+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:57:00.354852+0000","last_undegraded":"2024-05-09T08:57:00.354852+0000","last_fullsized":"2024-05-09T08:57:00.354852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349741+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:56:58.349741+0000","last_peered":"2024-05-09T08:56:58.349741+0000","last_clean":"2024-05-09T08:56:58.349741+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:56:58.349741+0000","last_undegraded":"2024-05-09T08:56:58.349741+0000","last_fullsized":"2024-05-09T08:56:58.349741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349950+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:56:59.349950+0000","last_peered":"2024-05-09T08:56:59.349950+0000","last_clean":"2024-05-09T08:56:59.349950+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:56:59.349950+0000","last_undegraded":"2024-05-09T08:56:59.349950+0000","last_fullsized":"2024-05-09T08:56:59.349950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350040+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:56:59.350040+0000","last_peered":"2024-05-09T08:56:59.350040+0000","last_clean":"2024-05-09T08:56:59.350040+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:56:59.350040+0000","last_undegraded":"2024-05-09T08:56:59.350040+0000","last_fullsized":"2024-05-09T08:56:59.350040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355709+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:57:00.355709+0000","last_peered":"2024-05-09T08:57:00.355709+0000","last_clean":"2024-05-09T08:57:00.355709+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:57:00.355709+0000","last_undegraded":"2024-05-09T08:57:00.355709+0000","last_fullsized":"2024-05-09T08:57:00.355709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356074+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:57:00.356074+0000","last_peered":"2024-05-09T08:57:00.356074+0000","last_clean":"2024-05-09T08:57:00.356074+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:57:00.356074+0000","last_undegraded":"2024-05-09T08:57:00.356074+0000","last_fullsized":"2024-05-09T08:57:00.356074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354228+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:57:00.354228+0000","last_peered":"2024-05-09T08:57:00.354228+0000","last_clean":"2024-05-09T08:57:00.354228+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:57:00.354228+0000","last_undegraded":"2024-05-09T08:57:00.354228+0000","last_fullsized":"2024-05-09T08:57:00.354228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'14","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349604+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:56:58.349604+0000","last_peered":"2024-05-09T08:56:58.349604+0000","last_clean":"2024-05-09T08:56:58.349604+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:56:58.349604+0000","last_undegraded":"2024-05-09T08:56:58.349604+0000","last_fullsized":"2024-05-09T08:56:58.349604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4760,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350129+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:56:58.350129+0000","last_peered":"2024-05-09T08:56:58.350129+0000","last_clean":"2024-05-09T08:56:58.350129+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:56:58.350129+0000","last_undegraded":"2024-05-09T08:56:58.350129+0000","last_fullsized":"2024-05-09T08:56:58.350129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355909+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:57:00.355909+0000","last_peered":"2024-05-09T08:57:00.355909+0000","last_clean":"2024-05-09T08:57:00.355909+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:57:00.355909+0000","last_undegraded":"2024-05-09T08:57:00.355909+0000","last_fullsized":"2024-05-09T08:57:00.355909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355105+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:57:00.355105+0000","last_peered":"2024-05-09T08:57:00.355105+0000","last_clean":"2024-05-09T08:57:00.355105+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:57:00.355105+0000","last_undegraded":"2024-05-09T08:57:00.355105+0000","last_fullsized":"2024-05-09T08:57:00.355105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350158+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:56:59.350158+0000","last_peered":"2024-05-09T08:56:59.350158+0000","last_clean":"2024-05-09T08:56:59.350158+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:56:59.350158+0000","last_undegraded":"2024-05-09T08:56:59.350158+0000","last_fullsized":"2024-05-09T08:56:59.350158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349654+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:56:58.349654+0000","last_peered":"2024-05-09T08:56:58.349654+0000","last_clean":"2024-05-09T08:56:58.349654+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:56:58.349654+0000","last_undegraded":"2024-05-09T08:56:58.349654+0000","last_fullsized":"2024-05-09T08:56:58.349654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350022+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:56:58.350022+0000","last_peered":"2024-05-09T08:56:58.350022+0000","last_clean":"2024-05-09T08:56:58.350022+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:56:58.350022+0000","last_undegraded":"2024-05-09T08:56:58.350022+0000","last_fullsized":"2024-05-09T08:56:58.350022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349650+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:56:58.349650+0000","last_peered":"2024-05-09T08:56:58.349650+0000","last_clean":"2024-05-09T08:56:58.349650+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:56:58.349650+0000","last_undegraded":"2024-05-09T08:56:58.349650+0000","last_fullsized":"2024-05-09T08:56:58.349650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349239+0000","last_change":"2024-05-09T08:56:00.689042+0000","last_active":"2024-05-09T08:56:58.349239+0000","last_peered":"2024-05-09T08:56:58.349239+0000","last_clean":"2024-05-09T08:56:58.349239+0000","last_became_active":"2024-05-09T08:56:00.688259+0000","last_became_peered":"2024-05-09T08:56:00.688259+0000","last_unstale":"2024-05-09T08:56:58.349239+0000","last_undegraded":"2024-05-09T08:56:58.349239+0000","last_fullsized":"2024-05-09T08:56:58.349239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:24.017474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349447+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:56:58.349447+0000","last_peered":"2024-05-09T08:56:58.349447+0000","last_clean":"2024-05-09T08:56:58.349447+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:56:58.349447+0000","last_undegraded":"2024-05-09T08:56:58.349447+0000","last_fullsized":"2024-05-09T08:56:58.349447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354194+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:57:00.354194+0000","last_peered":"2024-05-09T08:57:00.354194+0000","last_clean":"2024-05-09T08:57:00.354194+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:57:00.354194+0000","last_undegraded":"2024-05-09T08:57:00.354194+0000","last_fullsized":"2024-05-09T08:57:00.354194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349661+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:56:58.349661+0000","last_peered":"2024-05-09T08:56:58.349661+0000","last_clean":"2024-05-09T08:56:58.349661+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:56:58.349661+0000","last_undegraded":"2024-05-09T08:56:58.349661+0000","last_fullsized":"2024-05-09T08:56:58.349661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349933+0000","last_change":"2024-05-09T08:56:49.302708+0000","last_active":"2024-05-09T08:56:59.349933+0000","last_peered":"2024-05-09T08:56:59.349933+0000","last_clean":"2024-05-09T08:56:59.349933+0000","last_became_active":"2024-05-09T08:56:49.302353+0000","last_became_peered":"2024-05-09T08:56:49.302353+0000","last_unstale":"2024-05-09T08:56:59.349933+0000","last_undegraded":"2024-05-09T08:56:59.349933+0000","last_fullsized":"2024-05-09T08:56:59.349933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:33.894304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349885+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:56:58.349885+0000","last_peered":"2024-05-09T08:56:58.349885+0000","last_clean":"2024-05-09T08:56:58.349885+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:56:58.349885+0000","last_undegraded":"2024-05-09T08:56:58.349885+0000","last_fullsized":"2024-05-09T08:56:58.349885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356249+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:57:00.356249+0000","last_peered":"2024-05-09T08:57:00.356249+0000","last_clean":"2024-05-09T08:57:00.356249+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:57:00.356249+0000","last_undegraded":"2024-05-09T08:57:00.356249+0000","last_fullsized":"2024-05-09T08:57:00.356249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349720+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:56:59.349720+0000","last_peered":"2024-05-09T08:56:59.349720+0000","last_clean":"2024-05-09T08:56:59.349720+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:56:59.349720+0000","last_undegraded":"2024-05-09T08:56:59.349720+0000","last_fullsized":"2024-05-09T08:56:59.349720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349603+0000","last_change":"2024-05-09T08:56:00.685237+0000","last_active":"2024-05-09T08:56:58.349603+0000","last_peered":"2024-05-09T08:56:58.349603+0000","last_clean":"2024-05-09T08:56:58.349603+0000","last_became_active":"2024-05-09T08:56:00.684811+0000","last_became_peered":"2024-05-09T08:56:00.684811+0000","last_unstale":"2024-05-09T08:56:58.349603+0000","last_undegraded":"2024-05-09T08:56:58.349603+0000","last_fullsized":"2024-05-09T08:56:58.349603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:38:52.282840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350159+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:56:58.350159+0000","last_peered":"2024-05-09T08:56:58.350159+0000","last_clean":"2024-05-09T08:56:58.350159+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:56:58.350159+0000","last_undegraded":"2024-05-09T08:56:58.350159+0000","last_fullsized":"2024-05-09T08:56:58.350159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349406+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:56:58.349406+0000","last_peered":"2024-05-09T08:56:58.349406+0000","last_clean":"2024-05-09T08:56:58.349406+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:56:58.349406+0000","last_undegraded":"2024-05-09T08:56:58.349406+0000","last_fullsized":"2024-05-09T08:56:58.349406+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349597+0000","last_change":"2024-05-09T08:56:00.681821+0000","last_active":"2024-05-09T08:56:59.349597+0000","last_peered":"2024-05-09T08:56:59.349597+0000","last_clean":"2024-05-09T08:56:59.349597+0000","last_became_active":"2024-05-09T08:56:00.681536+0000","last_became_peered":"2024-05-09T08:56:00.681536+0000","last_unstale":"2024-05-09T08:56:59.349597+0000","last_undegraded":"2024-05-09T08:56:59.349597+0000","last_fullsized":"2024-05-09T08:56:59.349597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:55:29.522540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349914+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:56:59.349914+0000","last_peered":"2024-05-09T08:56:59.349914+0000","last_clean":"2024-05-09T08:56:59.349914+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:56:59.349914+0000","last_undegraded":"2024-05-09T08:56:59.349914+0000","last_fullsized":"2024-05-09T08:56:59.349914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349296+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:56:58.349296+0000","last_peered":"2024-05-09T08:56:58.349296+0000","last_clean":"2024-05-09T08:56:58.349296+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:56:58.349296+0000","last_undegraded":"2024-05-09T08:56:58.349296+0000","last_fullsized":"2024-05-09T08:56:58.349296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355728+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:57:00.355728+0000","last_peered":"2024-05-09T08:57:00.355728+0000","last_clean":"2024-05-09T08:57:00.355728+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:57:00.355728+0000","last_undegraded":"2024-05-09T08:57:00.355728+0000","last_fullsized":"2024-05-09T08:57:00.355728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349995+0000","last_change":"2024-05-09T08:56:00.693236+0000","last_active":"2024-05-09T08:56:58.349995+0000","last_peered":"2024-05-09T08:56:58.349995+0000","last_clean":"2024-05-09T08:56:58.349995+0000","last_became_active":"2024-05-09T08:56:00.692911+0000","last_became_peered":"2024-05-09T08:56:00.692911+0000","last_unstale":"2024-05-09T08:56:58.349995+0000","last_undegraded":"2024-05-09T08:56:58.349995+0000","last_fullsized":"2024-05-09T08:56:58.349995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:03.380535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355327+0000","last_change":"2024-05-09T08:56:01.688507+0000","last_active":"2024-05-09T08:57:00.355327+0000","last_peered":"2024-05-09T08:57:00.355327+0000","last_clean":"2024-05-09T08:57:00.355327+0000","last_became_active":"2024-05-09T08:56:01.688213+0000","last_became_peered":"2024-05-09T08:56:01.688213+0000","last_unstale":"2024-05-09T08:57:00.355327+0000","last_undegraded":"2024-05-09T08:57:00.355327+0000","last_fullsized":"2024-05-09T08:57:00.355327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:05:50.378284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350216+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:56:59.350216+0000","last_peered":"2024-05-09T08:56:59.350216+0000","last_clean":"2024-05-09T08:56:59.350216+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:56:59.350216+0000","last_undegraded":"2024-05-09T08:56:59.350216+0000","last_fullsized":"2024-05-09T08:56:59.350216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354167+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:57:00.354167+0000","last_peered":"2024-05-09T08:57:00.354167+0000","last_clean":"2024-05-09T08:57:00.354167+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:57:00.354167+0000","last_undegraded":"2024-05-09T08:57:00.354167+0000","last_fullsized":"2024-05-09T08:57:00.354167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356231+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:57:00.356231+0000","last_peered":"2024-05-09T08:57:00.356231+0000","last_clean":"2024-05-09T08:57:00.356231+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:57:00.356231+0000","last_undegraded":"2024-05-09T08:57:00.356231+0000","last_fullsized":"2024-05-09T08:57:00.356231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350132+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:56:59.350132+0000","last_peered":"2024-05-09T08:56:59.350132+0000","last_clean":"2024-05-09T08:56:59.350132+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:56:59.350132+0000","last_undegraded":"2024-05-09T08:56:59.350132+0000","last_fullsized":"2024-05-09T08:56:59.350132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350337+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:56:58.350337+0000","last_peered":"2024-05-09T08:56:58.350337+0000","last_clean":"2024-05-09T08:56:58.350337+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:56:58.350337+0000","last_undegraded":"2024-05-09T08:56:58.350337+0000","last_fullsized":"2024-05-09T08:56:58.350337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355083+0000","last_change":"2024-05-09T08:56:01.687562+0000","last_active":"2024-05-09T08:57:00.355083+0000","last_peered":"2024-05-09T08:57:00.355083+0000","last_clean":"2024-05-09T08:57:00.355083+0000","last_became_active":"2024-05-09T08:56:01.687236+0000","last_became_peered":"2024-05-09T08:56:01.687236+0000","last_unstale":"2024-05-09T08:57:00.355083+0000","last_undegraded":"2024-05-09T08:57:00.355083+0000","last_fullsized":"2024-05-09T08:57:00.355083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:30:40.352289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'8","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356385+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:57:00.356385+0000","last_peered":"2024-05-09T08:57:00.356385+0000","last_clean":"2024-05-09T08:57:00.356385+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:57:00.356385+0000","last_undegraded":"2024-05-09T08:57:00.356385+0000","last_fullsized":"2024-05-09T08:57:00.356385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350358+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:56:59.350358+0000","last_peered":"2024-05-09T08:56:59.350358+0000","last_clean":"2024-05-09T08:56:59.350358+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:56:59.350358+0000","last_undegraded":"2024-05-09T08:56:59.350358+0000","last_fullsized":"2024-05-09T08:56:59.350358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349848+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:56:58.349848+0000","last_peered":"2024-05-09T08:56:58.349848+0000","last_clean":"2024-05-09T08:56:58.349848+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:56:58.349848+0000","last_undegraded":"2024-05-09T08:56:58.349848+0000","last_fullsized":"2024-05-09T08:56:58.349848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355915+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:57:00.355915+0000","last_peered":"2024-05-09T08:57:00.355915+0000","last_clean":"2024-05-09T08:57:00.355915+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:57:00.355915+0000","last_undegraded":"2024-05-09T08:57:00.355915+0000","last_fullsized":"2024-05-09T08:57:00.355915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349171+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:56:58.349171+0000","last_peered":"2024-05-09T08:56:58.349171+0000","last_clean":"2024-05-09T08:56:58.349171+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:56:58.349171+0000","last_undegraded":"2024-05-09T08:56:58.349171+0000","last_fullsized":"2024-05-09T08:56:58.349171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.354044+0000","last_change":"2024-05-09T08:56:01.687697+0000","last_active":"2024-05-09T08:57:00.354044+0000","last_peered":"2024-05-09T08:57:00.354044+0000","last_clean":"2024-05-09T08:57:00.354044+0000","last_became_active":"2024-05-09T08:56:01.687407+0000","last_became_peered":"2024-05-09T08:56:01.687407+0000","last_unstale":"2024-05-09T08:57:00.354044+0000","last_undegraded":"2024-05-09T08:57:00.354044+0000","last_fullsized":"2024-05-09T08:57:00.354044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:52:45.837368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355180+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:57:00.355180+0000","last_peered":"2024-05-09T08:57:00.355180+0000","last_clean":"2024-05-09T08:57:00.355180+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:57:00.355180+0000","last_undegraded":"2024-05-09T08:57:00.355180+0000","last_fullsized":"2024-05-09T08:57:00.355180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349749+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:56:59.349749+0000","last_peered":"2024-05-09T08:56:59.349749+0000","last_clean":"2024-05-09T08:56:59.349749+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:56:59.349749+0000","last_undegraded":"2024-05-09T08:56:59.349749+0000","last_fullsized":"2024-05-09T08:56:59.349749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355638+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:57:00.355638+0000","last_peered":"2024-05-09T08:57:00.355638+0000","last_clean":"2024-05-09T08:57:00.355638+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:57:00.355638+0000","last_undegraded":"2024-05-09T08:57:00.355638+0000","last_fullsized":"2024-05-09T08:57:00.355638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350075+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:56:59.350075+0000","last_peered":"2024-05-09T08:56:59.350075+0000","last_clean":"2024-05-09T08:56:59.350075+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:56:59.350075+0000","last_undegraded":"2024-05-09T08:56:59.350075+0000","last_fullsized":"2024-05-09T08:56:59.350075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349787+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:56:58.349787+0000","last_peered":"2024-05-09T08:56:58.349787+0000","last_clean":"2024-05-09T08:56:58.349787+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:56:58.349787+0000","last_undegraded":"2024-05-09T08:56:58.349787+0000","last_fullsized":"2024-05-09T08:56:58.349787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350021+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:56:58.350021+0000","last_peered":"2024-05-09T08:56:58.350021+0000","last_clean":"2024-05-09T08:56:58.350021+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:56:58.350021+0000","last_undegraded":"2024-05-09T08:56:58.350021+0000","last_fullsized":"2024-05-09T08:56:58.350021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349575+0000","last_change":"2024-05-09T08:56:50.303865+0000","last_active":"2024-05-09T08:56:58.349575+0000","last_peered":"2024-05-09T08:56:58.349575+0000","last_clean":"2024-05-09T08:56:58.349575+0000","last_became_active":"2024-05-09T08:56:50.303517+0000","last_became_peered":"2024-05-09T08:56:50.303517+0000","last_unstale":"2024-05-09T08:56:58.349575+0000","last_undegraded":"2024-05-09T08:56:58.349575+0000","last_fullsized":"2024-05-09T08:56:58.349575+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:17:17.229833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.349425+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:56:58.349425+0000","last_peered":"2024-05-09T08:56:58.349425+0000","last_clean":"2024-05-09T08:56:58.349425+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:56:58.349425+0000","last_undegraded":"2024-05-09T08:56:58.349425+0000","last_fullsized":"2024-05-09T08:56:58.349425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349631+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:56:59.349631+0000","last_peered":"2024-05-09T08:56:59.349631+0000","last_clean":"2024-05-09T08:56:59.349631+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:56:59.349631+0000","last_undegraded":"2024-05-09T08:56:59.349631+0000","last_fullsized":"2024-05-09T08:56:59.349631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T08:56:58.350248+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:56:58.350248+0000","last_peered":"2024-05-09T08:56:58.350248+0000","last_clean":"2024-05-09T08:56:58.350248+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:56:58.350248+0000","last_undegraded":"2024-05-09T08:56:58.350248+0000","last_fullsized":"2024-05-09T08:56:58.350248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.355604+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:57:00.355604+0000","last_peered":"2024-05-09T08:57:00.355604+0000","last_clean":"2024-05-09T08:57:00.355604+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:57:00.355604+0000","last_undegraded":"2024-05-09T08:57:00.355604+0000","last_fullsized":"2024-05-09T08:57:00.355604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.350056+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:56:59.350056+0000","last_peered":"2024-05-09T08:56:59.350056+0000","last_clean":"2024-05-09T08:56:59.350056+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:56:59.350056+0000","last_undegraded":"2024-05-09T08:56:59.350056+0000","last_fullsized":"2024-05-09T08:56:59.350056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T08:56:59.349821+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:56:59.349821+0000","last_peered":"2024-05-09T08:56:59.349821+0000","last_clean":"2024-05-09T08:56:59.349821+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:56:59.349821+0000","last_undegraded":"2024-05-09T08:56:59.349821+0000","last_fullsized":"2024-05-09T08:56:59.349821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-05-09T08:57:00.356003+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:57:00.356003+0000","last_peered":"2024-05-09T08:57:00.356003+0000","last_clean":"2024-05-09T08:57:00.356003+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:57:00.356003+0000","last_undegraded":"2024-05-09T08:57:00.356003+0000","last_fullsized":"2024-05-09T08:57:00.356003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11438,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":215,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":187,"ondisk_log_size":187,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":730061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":142887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":147334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":144732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":385024,"data_stored":149242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":372736,"data_stored":138799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":138869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":728579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":5550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:57:02.074 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-09T08:57:02.074 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:02.074 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-05-09T08:57:02.074 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-05-09T08:57:02.113 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:02.114+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:02.160 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.161 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:02.185 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:02.186+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.218+0000 7f620d116640 1 Processor -- start 2024-05-09T08:57:02.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.218+0000 7f620d116640 1 -- start start 2024-05-09T08:57:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.219+0000 7f620d116640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6208154b90 0x7f62080cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.219+0000 7f620d116640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6208072600 con 0x7f6208154f70 2024-05-09T08:57:02.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.219+0000 7f620d116640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62080728e0 con 0x7f62080c6d50 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.219+0000 7f620d116640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f62080c5880 con 0x7f6208154b90 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.219+0000 7f620659c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6208154f70 0x7f62080c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51264/0 (socket says 172.21.15.49:51264) 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f620659c640 1 -- 172.21.15.49:0/2062780971 learned_addr learned my addr 172.21.15.49:0/2062780971 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6206d9d640 1 --2- 172.21.15.49:0/2062780971 >> v2:172.21.15.183:3300/0 conn(0x7f6208154b90 0x7f62080cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6205d9b640 1 -- 172.21.15.49:0/2062780971 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 775256317 0 0) 0x7f6208072600 con 0x7f6208154f70 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6205d9b640 1 -- 172.21.15.49:0/2062780971 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61f0003520 con 0x7f6208154f70 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6206d9d640 1 -- 172.21.15.49:0/2062780971 >> v1:172.21.15.183:6789/0 conn(0x7f62080c6d50 legacy=0x7f6208071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6206d9d640 1 -- 172.21.15.49:0/2062780971 >> v1:172.21.15.49:6789/0 conn(0x7f6208154f70 legacy=0x7f62080c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.220+0000 7f6206d9d640 1 -- 172.21.15.49:0/2062780971 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62080c5b60 con 0x7f6208154b90 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f6206d9d640 1 --2- 172.21.15.49:0/2062780971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6208154b90 0x7f62080cb280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097a0 tx=0x7f61f802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b089685758292f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f6205d9b640 1 -- 172.21.15.49:0/2062780971 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8037070 con 0x7f6208154b90 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f6205d9b640 1 -- 172.21.15.49:0/2062780971 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f8032640 con 0x7f6208154b90 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f6205d9b640 1 -- 172.21.15.49:0/2062780971 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8032a10 con 0x7f6208154b90 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f620d116640 1 -- 172.21.15.49:0/2062780971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6208154b90 msgr2=0x7f62080cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f620d116640 1 --2- 172.21.15.49:0/2062780971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6208154b90 0x7f62080cb280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097a0 tx=0x7f61f802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f620d116640 1 -- 172.21.15.49:0/2062780971 shutdown_connections 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f620d116640 1 --2- 172.21.15.49:0/2062780971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6208154b90 0x7f62080cb280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.221+0000 7f620d116640 1 -- 172.21.15.49:0/2062780971 >> 172.21.15.49:0/2062780971 conn(0x7f6208074840 msgr2=0x7f6208074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.222+0000 7f620d116640 1 -- 172.21.15.49:0/2062780971 shutdown_connections 2024-05-09T08:57:02.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.222+0000 7f620d116640 1 -- 172.21.15.49:0/2062780971 wait complete. 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.223+0000 7f620d116640 1 Processor -- start 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.223+0000 7f620d116640 1 -- start start 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62080c6d50 0x7f62081e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6208154b90 0x7f62081f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 0x7f62081f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f620815af40 con 0x7f6208154f70 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62080c6120 con 0x7f62080c6d50 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620d116640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f62080cb7c0 con 0x7f6208154b90 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 0x7f62081f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 0x7f62081f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55788/0 (socket says 172.21.15.49:55788) 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 -- 172.21.15.49:0/1356702825 learned_addr learned my addr 172.21.15.49:0/1356702825 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f6206d9d640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62080c6d50 0x7f62081e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620659c640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6208154b90 0x7f62081f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 -- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6208154b90 msgr2=0x7f62081f0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6208154b90 0x7f62081f0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 -- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62080c6d50 msgr2=0x7f62081e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62080c6d50 0x7f62081e79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620814b950 con 0x7f6208154f70 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f6206d9d640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62080c6d50 0x7f62081e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.224+0000 7f620759e640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 0x7f62081f1170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f620000c7d0 tx=0x7f620000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.225+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200003b70 con 0x7f6208154f70 2024-05-09T08:57:02.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.225+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6200003d30 con 0x7f6208154f70 2024-05-09T08:57:02.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.225+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081f43b0 con 0x7f6208154f70 2024-05-09T08:57:02.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.225+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000aa90 con 0x7f6208154f70 2024-05-09T08:57:02.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.225+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081f4970 con 0x7f6208154f70 2024-05-09T08:57:02.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.226+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f620804f660 con 0x7f6208154f70 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.226+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f620000ac30 con 0x7f6208154f70 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f61ef7fe640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f61f406be30 0x7f61f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f620008af20 con 0x7f6208154f70 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f61ef7fe640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f61f40741b0 0x7f61f4076650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6200004ec0 con 0x7f61f40741b0 2024-05-09T08:57:02.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f6206d9d640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f61f406be30 0x7f61f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f620008d620 con 0x7f6208154f70 2024-05-09T08:57:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.227+0000 7f620659c640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f61f40741b0 0x7f61f4076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.228+0000 7f6206d9d640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f61f406be30 0x7f61f406e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61f8034000 tx=0x7f61f802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.228+0000 7f620659c640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f61f40741b0 0x7f61f4076650 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.228+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6200004ec0 con 0x7f61f40741b0 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.245+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f62081543b0 con 0x7f61f40741b0 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f61ef7fe640 1 -- 172.21.15.49:0/1356702825 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f62081543b0 con 0x7f61f40741b0 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f61f40741b0 msgr2=0x7f61f4076650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f61f40741b0 0x7f61f4076650 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f61f406be30 msgr2=0x7f61f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f61f406be30 0x7f61f406e2f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61f8034000 tx=0x7f61f802e000 comp rx=0 tx=0).stop 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 msgr2=0x7f62081f1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 --2- 172.21.15.49:0/1356702825 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6208154f70 0x7f62081f1170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f620000c7d0 tx=0x7f620000cc90 comp rx=0 tx=0).stop 2024-05-09T08:57:02.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620759e640 1 -- 172.21.15.49:0/1356702825 reap_dead start 2024-05-09T08:57:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 shutdown_connections 2024-05-09T08:57:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 >> 172.21.15.49:0/1356702825 conn(0x7f6208074840 msgr2=0x7f620815a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 shutdown_connections 2024-05-09T08:57:02.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.246+0000 7f620d116640 1 -- 172.21.15.49:0/1356702825 wait complete. 2024-05-09T08:57:02.256 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-05-09T08:57:02.283 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.283 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.286 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:02.287+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:02.293 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:02.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.363+0000 7fb0d409e640 1 Processor -- start 2024-05-09T08:57:02.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.363+0000 7fb0d409e640 1 -- start start 2024-05-09T08:57:02.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d409e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb0cc154f70 0x7fb0cc0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d409e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0cc0c6b80 0x7fb0cc071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d409e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0cc0c6fd0 con 0x7fb0cc154b90 2024-05-09T08:57:02.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d409e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc072600 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d409e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fb0cc0728e0 con 0x7fb0cc154f70 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d263b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fb0cc0c6b80 0x7fb0cc071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d1e3a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fb0cc154b90 0x7fb0cc0cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51278/0 (socket says 172.21.15.49:51278) 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.364+0000 7fb0d1e3a640 1 -- 172.21.15.49:0/3527399238 learned_addr learned my addr 172.21.15.49:0/3527399238 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.365+0000 7fb0d263b640 1 -- 172.21.15.49:0/3527399238 >> v2:172.21.15.183:3300/0 conn(0x7fb0cc154f70 msgr2=0x7fb0cc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.365+0000 7fb0d263b640 1 --2- 172.21.15.49:0/3527399238 >> v2:172.21.15.183:3300/0 conn(0x7fb0cc154f70 0x7fb0cc0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.365+0000 7fb0d263b640 1 -- 172.21.15.49:0/3527399238 >> v1:172.21.15.49:6789/0 conn(0x7fb0cc154b90 legacy=0x7fb0cc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.365+0000 7fb0d0e38640 1 -- 172.21.15.49:0/3527399238 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604172426 0 0) 0x7fb0cc0c6fd0 con 0x7fb0cc154b90 2024-05-09T08:57:02.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.365+0000 7fb0d263b640 1 -- 172.21.15.49:0/3527399238 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc0c5880 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d263b640 1 --2- 172.21.15.49:0/3527399238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc0c6b80 0x7fb0cc071820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0cdd20 tx=0x7fb0c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44408228bd41a5d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d0e38640 1 -- 172.21.15.49:0/3527399238 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4037070 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d0e38640 1 -- 172.21.15.49:0/3527399238 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c4032440 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d0e38640 1 -- 172.21.15.49:0/3527399238 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4032810 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 -- 172.21.15.49:0/3527399238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc0c6b80 msgr2=0x7fb0cc071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3527399238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc0c6b80 0x7fb0cc071820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0cdd20 tx=0x7fb0c402a230 comp rx=0 tx=0).stop 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 -- 172.21.15.49:0/3527399238 shutdown_connections 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3527399238 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc0c6b80 0x7fb0cc071820 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3527399238 >> v2:172.21.15.183:3300/0 conn(0x7fb0cc154f70 0x7fb0cc0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.366+0000 7fb0d409e640 1 -- 172.21.15.49:0/3527399238 >> 172.21.15.49:0/3527399238 conn(0x7fb0cc074840 msgr2=0x7fb0cc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.367+0000 7fb0d409e640 1 -- 172.21.15.49:0/3527399238 shutdown_connections 2024-05-09T08:57:02.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.367+0000 7fb0d409e640 1 -- 172.21.15.49:0/3527399238 wait complete. 2024-05-09T08:57:02.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 Processor -- start 2024-05-09T08:57:02.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 -- start start 2024-05-09T08:57:02.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0cc0c6b80 0x7fb0cc14f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc154b90 0x7fb0cc14f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 0x7fb0cc14ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0c6120 con 0x7fb0cc154f70 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc0c5e40 con 0x7fb0cc154b90 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.368+0000 7fb0d409e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fb0cc15af40 con 0x7fb0cc0c6b80 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 0x7fb0cc14ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 0x7fb0cc14ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55812/0 (socket says 172.21.15.49:55812) 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 -- 172.21.15.49:0/3768350628 learned_addr learned my addr 172.21.15.49:0/3768350628 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d1639640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc154b90 0x7fb0cc14f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d1e3a640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0cc0c6b80 0x7fb0cc14f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 -- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0cc0c6b80 msgr2=0x7fb0cc14f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0cc0c6b80 0x7fb0cc14f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 -- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc154b90 msgr2=0x7fb0cc14f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc154b90 0x7fb0cc14f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1f1240 con 0x7fb0cc154f70 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d263b640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 0x7fb0cc14ba70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0c402a820 tx=0x7fb0c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4041070 con 0x7fb0cc154f70 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c403c870 con 0x7fb0cc154f70 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc14b0a0 con 0x7fb0cc154f70 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.369+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4037040 con 0x7fb0cc154f70 2024-05-09T08:57:02.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.370+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc150c20 con 0x7fb0cc154f70 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.370+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0cc1592f0 con 0x7fb0cc154f70 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0c4044070 con 0x7fb0cc154f70 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb0b006be60 0x7fb0b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7fb0c40abb50 con 0x7fb0cc154f70 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0b0074240 0x7fb0b00766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0c4008eb0 con 0x7fb0b0074240 2024-05-09T08:57:02.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb0c40b02c0 con 0x7fb0cc154f70 2024-05-09T08:57:02.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0d1e3a640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb0b006be60 0x7fb0b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.372+0000 7fb0d1639640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0b0074240 0x7fb0b00766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:57:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.373+0000 7fb0d1639640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0b0074240 0x7fb0b00766e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.373+0000 7fb0d1e3a640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb0b006be60 0x7fb0b006e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80012e0 tx=0x7fb0c8001630 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.373+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0c4008eb0 con 0x7fb0b0074240 2024-05-09T08:57:02.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.391+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fb0cc04f660 con 0x7fb0b0074240 2024-05-09T08:57:02.390 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:02.391+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.391+0000 7fb0c27fc640 1 -- 172.21.15.49:0/3768350628 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fb0cc04f660 con 0x7fb0b0074240 2024-05-09T08:57:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0b0074240 msgr2=0x7fb0b00766e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fb0b0074240 0x7fb0b00766e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb0b006be60 msgr2=0x7fb0b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fb0b006be60 0x7fb0b006e320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80012e0 tx=0x7fb0c8001630 comp rx=0 tx=0).stop 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 msgr2=0x7fb0cc14ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 --2- 172.21.15.49:0/3768350628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fb0cc154f70 0x7fb0cc14ba70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0c402a820 tx=0x7fb0c40046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d263b640 1 -- 172.21.15.49:0/3768350628 reap_dead start 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 shutdown_connections 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 >> 172.21.15.49:0/3768350628 conn(0x7fb0cc074840 msgr2=0x7fb0cc1e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 shutdown_connections 2024-05-09T08:57:02.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.392+0000 7fb0d409e640 1 -- 172.21.15.49:0/3768350628 wait complete. 2024-05-09T08:57:02.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-05-09T08:57:02.451 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.451 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.459 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:02.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.484+0000 7f468769a640 1 Processor -- start 2024-05-09T08:57:02.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.484+0000 7f468769a640 1 -- start start 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f468769a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f46800cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f468769a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46800c6d50 0x7f46800cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f468769a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4680072600 con 0x7f4680154f70 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f468769a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f46800728e0 con 0x7f46800c6d50 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f468769a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f46800c5880 con 0x7f4680154b90 2024-05-09T08:57:02.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f4685436640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f46800cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f4685c37640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46800c6d50 0x7f46800cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f4684c35640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4680154f70 0x7f46800c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49644/0 (socket says 172.21.15.49:49644) 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.485+0000 7f4684c35640 1 -- 172.21.15.49:0/3383731390 learned_addr learned my addr 172.21.15.49:0/3383731390 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685c37640 1 -- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 msgr2=0x7f46800cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685c37640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f46800cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685c37640 1 -- 172.21.15.49:0/3383731390 >> v1:172.21.15.183:6789/0 conn(0x7f4680154f70 legacy=0x7f46800c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685c37640 1 -- 172.21.15.49:0/3383731390 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800c5b60 con 0x7f46800c6d50 2024-05-09T08:57:02.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f467ffff640 1 -- 172.21.15.49:0/3383731390 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3313034862 0 0) 0x7f4680072600 con 0x7f4680154f70 2024-05-09T08:57:02.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685436640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f46800cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.486+0000 7f4685c37640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f46800cb7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f467002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15e96778c5a5e5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f467ffff640 1 -- 172.21.15.49:0/3383731390 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670007740 con 0x7f46800c6d50 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f467ffff640 1 -- 172.21.15.49:0/3383731390 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46700078e0 con 0x7f46800c6d50 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f467ffff640 1 -- 172.21.15.49:0/3383731390 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670007cb0 con 0x7f46800c6d50 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 -- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 msgr2=0x7f46800cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f46800cb7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f467002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 -- 172.21.15.49:0/3383731390 shutdown_connections 2024-05-09T08:57:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f46800cb7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 --2- 172.21.15.49:0/3383731390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f46800cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 -- 172.21.15.49:0/3383731390 >> 172.21.15.49:0/3383731390 conn(0x7f4680074840 msgr2=0x7f4680074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 -- 172.21.15.49:0/3383731390 shutdown_connections 2024-05-09T08:57:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.487+0000 7f468769a640 1 -- 172.21.15.49:0/3383731390 wait complete. 2024-05-09T08:57:02.490 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 Processor -- start 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 -- start start 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f468014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f468014c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f468015af40 con 0x7f4680154f70 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46800c6120 con 0x7f46800c6d50 2024-05-09T08:57:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.489+0000 7f468769a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4680071820 con 0x7f4680154b90 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55820/0 (socket says 172.21.15.49:55820) 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 -- 172.21.15.49:0/2399931212 learned_addr learned my addr 172.21.15.49:0/2399931212 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685436640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f468014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4684c35640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f468014c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 -- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 msgr2=0x7f468014c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f468014c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 -- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 msgr2=0x7f468014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f468014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 -- 172.21.15.49:0/2399931212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46800c83e0 con 0x7f4680154f70 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685436640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f468014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f4685c37640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f4670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670040070 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4670031c40 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46800c8680 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.490+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670009450 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.491+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46800c8c20 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.492+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468004f660 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.492+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4670039040 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.493+0000 7f467dffb640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 0x7f466c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.493+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f46700abe40 con 0x7f4680154f70 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.493+0000 7f4685436640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 0x7f466c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.493+0000 7f4685436640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 0x7f466c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4678009ba0 tx=0x7f4678008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.501+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f467007a6d0 con 0x7f4680154f70 2024-05-09T08:57:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.555+0000 7fe014a93640 1 Processor -- start 2024-05-09T08:57:02.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.555+0000 7fe014a93640 1 -- start start 2024-05-09T08:57:02.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe014a93640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe010154b90 0x7fe0100cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe014a93640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe010154f70 0x7fe0100cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe014a93640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe010072600 con 0x7fe0100c6640 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe014a93640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100728e0 con 0x7fe010154b90 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe014a93640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100c5880 con 0x7fe010154f70 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe010154b90 0x7fe0100cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe010154b90 0x7fe0100cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55832/0 (socket says 172.21.15.49:55832) 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 -- 172.21.15.49:0/1910871945 learned_addr learned my addr 172.21.15.49:0/1910871945 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00dd9b640 1 --2- 172.21.15.49:0/1910871945 >> v2:172.21.15.183:3300/0 conn(0x7fe010154f70 0x7fe0100cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 -- 172.21.15.49:0/1910871945 >> v2:172.21.15.183:3300/0 conn(0x7fe010154f70 msgr2=0x7fe0100cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 --2- 172.21.15.49:0/1910871945 >> v2:172.21.15.183:3300/0 conn(0x7fe010154f70 0x7fe0100cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 -- 172.21.15.49:0/1910871945 >> v1:172.21.15.183:6789/0 conn(0x7fe0100c6640 legacy=0x7fe0100c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.557+0000 7fe00e59c640 1 -- 172.21.15.49:0/1910871945 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100c5b60 con 0x7fe010154b90 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.558+0000 7fe00e59c640 1 --2- 172.21.15.49:0/1910871945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154b90 0x7fe0100cb280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0100cdd20 tx=0x7fe00002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7d701762a13acd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.558+0000 7fe00d59a640 1 -- 172.21.15.49:0/1910871945 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000037070 con 0x7fe010154b90 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.558+0000 7fe00d59a640 1 -- 172.21.15.49:0/1910871945 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00002ae50 con 0x7fe010154b90 2024-05-09T08:57:02.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.558+0000 7fe00d59a640 1 -- 172.21.15.49:0/1910871945 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000032860 con 0x7fe010154b90 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 -- 172.21.15.49:0/1910871945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154b90 msgr2=0x7fe0100cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 --2- 172.21.15.49:0/1910871945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154b90 0x7fe0100cb280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0100cdd20 tx=0x7fe00002a300 comp rx=0 tx=0).stop 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 -- 172.21.15.49:0/1910871945 shutdown_connections 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 --2- 172.21.15.49:0/1910871945 >> v2:172.21.15.183:3300/0 conn(0x7fe010154f70 0x7fe0100cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 --2- 172.21.15.49:0/1910871945 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154b90 0x7fe0100cb280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 -- 172.21.15.49:0/1910871945 >> 172.21.15.49:0/1910871945 conn(0x7fe010074840 msgr2=0x7fe010074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 -- 172.21.15.49:0/1910871945 shutdown_connections 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.559+0000 7fe014a93640 1 -- 172.21.15.49:0/1910871945 wait complete. 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.560+0000 7fe014a93640 1 Processor -- start 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.560+0000 7fe014a93640 1 -- start start 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0100c6640 0x7fe0100d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe010154b90 0x7fe01014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 0x7fe01014d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe01015af40 con 0x7fe010154f70 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100c6120 con 0x7fe010154b90 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe014a93640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe010071820 con 0x7fe0100c6640 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 0x7fe01014d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe00ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 0x7fe01014d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55842/0 (socket says 172.21.15.49:55842) 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe00ed9d640 1 -- 172.21.15.49:0/352110379 learned_addr learned my addr 172.21.15.49:0/352110379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe00e59c640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0100c6640 0x7fe0100d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.561+0000 7fe00dd9b640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe010154b90 0x7fe01014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 -- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0100c6640 msgr2=0x7fe0100d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe0100c6640 0x7fe0100d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 -- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe010154b90 msgr2=0x7fe01014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe010154b90 0x7fe01014cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0101f0980 con 0x7fe010154f70 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe00ed9d640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 0x7fe01014d0b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c7d0 tx=0x7fe00800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008003b70 con 0x7fe010154f70 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe008003d30 con 0x7fe010154f70 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0101f0c60 con 0x7fe010154f70 2024-05-09T08:57:02.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00800aa90 con 0x7fe010154f70 2024-05-09T08:57:02.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.562+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0101f1220 con 0x7fe010154f70 2024-05-09T08:57:02.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.563+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe01014c750 con 0x7fe010154f70 2024-05-09T08:57:02.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe00800ac30 con 0x7fe010154f70 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdff006be60 0x7fdff006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7fe00808ac60 con 0x7fe010154f70 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fdff00741e0 0x7fdff0076680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe008010e50 con 0x7fdff00741e0 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe008090070 con 0x7fe010154f70 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe00e59c640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdff006be60 0x7fdff006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.564+0000 7fe00dd9b640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fdff00741e0 0x7fdff0076680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.565+0000 7fe00e59c640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdff006be60 0x7fdff006e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe000034000 tx=0x7fe00002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.565+0000 7fe00dd9b640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fdff00741e0 0x7fdff0076680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.565+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe008010e50 con 0x7fdff00741e0 2024-05-09T08:57:02.575 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.575 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.603+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fe01004f660 con 0x7fdff00741e0 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.603+0000 7fe006ffd640 1 -- 172.21.15.49:0/352110379 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fe01004f660 con 0x7fdff00741e0 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fdff00741e0 msgr2=0x7fdff0076680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fdff00741e0 0x7fdff0076680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdff006be60 msgr2=0x7fdff006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdff006be60 0x7fdff006e320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe000034000 tx=0x7fe00002e000 comp rx=0 tx=0).stop 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 msgr2=0x7fe01014d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 --2- 172.21.15.49:0/352110379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe010154f70 0x7fe01014d0b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c7d0 tx=0x7fe00800cc90 comp rx=0 tx=0).stop 2024-05-09T08:57:02.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe00ed9d640 1 -- 172.21.15.49:0/352110379 reap_dead start 2024-05-09T08:57:02.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 shutdown_connections 2024-05-09T08:57:02.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 >> 172.21.15.49:0/352110379 conn(0x7fe010074840 msgr2=0x7fe010153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 shutdown_connections 2024-05-09T08:57:02.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.604+0000 7fe014a93640 1 -- 172.21.15.49:0/352110379 wait complete. 2024-05-09T08:57:02.614 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-05-09T08:57:02.628 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:02.629+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:02.688 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.688 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.689 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:02.690+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:02.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:02.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.721+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f468015bbf0 con 0x7f4680154f70 2024-05-09T08:57:02.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.760+0000 7f64aa5f9640 1 Processor -- start 2024-05-09T08:57:02.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.760+0000 7f64aa5f9640 1 -- start start 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64aa5f9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64a4155010 0x7f64a40ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64aa5f9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a41553f0 con 0x7f64a415e310 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64aa5f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a40729a0 con 0x7f64a40c9c80 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64aa5f9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f64a40cad60 con 0x7f64a4155010 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a37fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f64a415e310 0x7f64a40caf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43894/0 (socket says 172.21.15.49:43894) 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a37fe640 1 -- 172.21.15.49:0/4169426951 learned_addr learned my addr 172.21.15.49:0/4169426951 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a3fff640 1 --2- 172.21.15.49:0/4169426951 >> v2:172.21.15.183:3300/0 conn(0x7f64a4155010 0x7f64a40ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 563861662 0 0) 0x7f64a41553f0 con 0x7f64a415e310 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f648c003520 con 0x7f64a415e310 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883384090 0 0) 0x7f64a40729a0 con 0x7f64a40c9c80 2024-05-09T08:57:02.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64a41553f0 con 0x7f64a40c9c80 2024-05-09T08:57:02.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a3fff640 1 -- 172.21.15.49:0/4169426951 >> v1:172.21.15.183:6789/0 conn(0x7f64a40c9c80 legacy=0x7f64a41656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a3fff640 1 -- 172.21.15.49:0/4169426951 >> v1:172.21.15.49:6789/0 conn(0x7f64a415e310 legacy=0x7f64a40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.761+0000 7f64a3fff640 1 -- 172.21.15.49:0/4169426951 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4072c80 con 0x7f64a4155010 2024-05-09T08:57:02.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.762+0000 7f64a3fff640 1 --2- 172.21.15.49:0/4169426951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a4155010 0x7f64a40ca820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64940097a0 tx=0x7f649402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a536f160df82414 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.762+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494037070 con 0x7f64a4155010 2024-05-09T08:57:02.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.762+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494032440 con 0x7f64a4155010 2024-05-09T08:57:02.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.762+0000 7f64a2ffd640 1 -- 172.21.15.49:0/4169426951 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494032810 con 0x7f64a4155010 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 -- 172.21.15.49:0/4169426951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a4155010 msgr2=0x7f64a40ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 --2- 172.21.15.49:0/4169426951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a4155010 0x7f64a40ca820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f64940097a0 tx=0x7f649402a230 comp rx=0 tx=0).stop 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 -- 172.21.15.49:0/4169426951 shutdown_connections 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 --2- 172.21.15.49:0/4169426951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a4155010 0x7f64a40ca820 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 -- 172.21.15.49:0/4169426951 >> 172.21.15.49:0/4169426951 conn(0x7f64a4067d90 msgr2=0x7f64a4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 -- 172.21.15.49:0/4169426951 shutdown_connections 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.763+0000 7f64aa5f9640 1 -- 172.21.15.49:0/4169426951 wait complete. 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.764+0000 7f64aa5f9640 1 Processor -- start 2024-05-09T08:57:02.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.764+0000 7f64aa5f9640 1 -- start start 2024-05-09T08:57:02.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 0x7f64a41e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64a4155010 0x7f64a41f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a415e310 0x7f64a40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f64a40ca300 con 0x7f64a40c9c80 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f64a4073240 con 0x7f64a415e310 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64aa5f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f64a4073760 con 0x7f64a4155010 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 0x7f64a41e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 0x7f64a41e78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49820/0 (socket says 172.21.15.49:49820) 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64a3fff640 1 -- 172.21.15.49:0/885179985 learned_addr learned my addr 172.21.15.49:0/885179985 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64a8b96640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a415e310 0x7f64a40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.765+0000 7f64a37fe640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64a4155010 0x7f64a41f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 -- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64a4155010 msgr2=0x7f64a41f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64a4155010 0x7f64a41f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 -- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a415e310 msgr2=0x7f64a40c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64a415e310 0x7f64a40c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a415f1e0 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a3fff640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 0x7f64a41e78d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6494035040 tx=0x7f64940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494041030 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f649403d4e0 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a41e7e10 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494037040 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.766+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41f4dc0 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.767+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64a404f660 con 0x7f64a40c9c80 2024-05-09T08:57:02.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.767+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6494044070 con 0x7f64a40c9c80 2024-05-09T08:57:02.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a0ff9640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f649006be30 0x7f649006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f64940abc70 con 0x7f64a40c9c80 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a37fe640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f649006be30 0x7f649006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a0ff9640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6490074210 0x7f64900766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6494008eb0 con 0x7f6490074210 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f64940ac100 con 0x7f64a40c9c80 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.768+0000 7f64a8b96640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6490074210 0x7f64900766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.769+0000 7f64a37fe640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f649006be30 0x7f649006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64a40748b0 tx=0x7f6498007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.769+0000 7f64a8b96640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6490074210 0x7f64900766b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.769+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6494008eb0 con 0x7f6490074210 2024-05-09T08:57:02.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.786+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f64a4158740 con 0x7f6490074210 2024-05-09T08:57:02.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.786+0000 7f64a0ff9640 1 -- 172.21.15.49:0/885179985 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f64a4158740 con 0x7f6490074210 2024-05-09T08:57:02.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6490074210 msgr2=0x7f64900766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6490074210 0x7f64900766b0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f649006be30 msgr2=0x7f649006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f649006be30 0x7f649006e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64a40748b0 tx=0x7f6498007d50 comp rx=0 tx=0).stop 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 msgr2=0x7f64a41e78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 --2- 172.21.15.49:0/885179985 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f64a40c9c80 0x7f64a41e78d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6494035040 tx=0x7f64940046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64a8b96640 1 -- 172.21.15.49:0/885179985 reap_dead start 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 shutdown_connections 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 >> 172.21.15.49:0/885179985 conn(0x7f64a4067d90 msgr2=0x7f64a415e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 shutdown_connections 2024-05-09T08:57:02.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.787+0000 7f64aa5f9640 1 -- 172.21.15.49:0/885179985 wait complete. 2024-05-09T08:57:02.790 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:02.791+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:02.796 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-05-09T08:57:02.839 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.840 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.848 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:02.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:02.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.903+0000 7f8509f22640 1 Processor -- start 2024-05-09T08:57:02.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.903+0000 7f8509f22640 1 -- start start 2024-05-09T08:57:02.902 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:02.903+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8509f22640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 0x7f85040c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8509f22640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 0x7f85040c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8509f22640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85040c3710 con 0x7f8504072a50 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8509f22640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f85040c34e0 con 0x7f8504152ea0 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8509f22640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8504067e50 con 0x7f85040c7340 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8503fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8504072a50 0x7f850415c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43898/0 (socket says 172.21.15.49:43898) 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8503fff640 1 -- 172.21.15.49:0/252741777 learned_addr learned my addr 172.21.15.49:0/252741777 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f85037fe640 1 --2- 172.21.15.49:0/252741777 >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 0x7f85040c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8502ffd640 1 --2- 172.21.15.49:0/252741777 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 0x7f85040c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f85027fc640 1 -- 172.21.15.49:0/252741777 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702382797 0 0) 0x7f85040c3710 con 0x7f8504072a50 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f85027fc640 1 -- 172.21.15.49:0/252741777 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84ec003520 con 0x7f8504072a50 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8502ffd640 1 -- 172.21.15.49:0/252741777 >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 msgr2=0x7f85040c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8502ffd640 1 --2- 172.21.15.49:0/252741777 >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 0x7f85040c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8502ffd640 1 -- 172.21.15.49:0/252741777 >> v1:172.21.15.49:6789/0 conn(0x7f8504072a50 legacy=0x7f850415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f8502ffd640 1 -- 172.21.15.49:0/252741777 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8504067890 con 0x7f85040c7340 2024-05-09T08:57:02.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.904+0000 7f85037fe640 1 --2- 172.21.15.49:0/252741777 >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 0x7f85040c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:02.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.905+0000 7f8502ffd640 1 --2- 172.21.15.49:0/252741777 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 0x7f85040c7ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84f80097a0 tx=0x7f84f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1db8f7e83e5402d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.905+0000 7f85027fc640 1 -- 172.21.15.49:0/252741777 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037070 con 0x7f85040c7340 2024-05-09T08:57:02.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.905+0000 7f85027fc640 1 -- 172.21.15.49:0/252741777 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8032440 con 0x7f85040c7340 2024-05-09T08:57:02.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.905+0000 7f85027fc640 1 -- 172.21.15.49:0/252741777 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8032810 con 0x7f85040c7340 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 -- 172.21.15.49:0/252741777 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 msgr2=0x7f85040c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 --2- 172.21.15.49:0/252741777 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 0x7f85040c7ae0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84f80097a0 tx=0x7f84f802a180 comp rx=0 tx=0).stop 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 -- 172.21.15.49:0/252741777 shutdown_connections 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 --2- 172.21.15.49:0/252741777 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85040c7340 0x7f85040c7ae0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 --2- 172.21.15.49:0/252741777 >> v2:172.21.15.183:3300/0 conn(0x7f8504152ea0 0x7f85040c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 -- 172.21.15.49:0/252741777 >> 172.21.15.49:0/252741777 conn(0x7f85040c9440 msgr2=0x7f85040c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 -- 172.21.15.49:0/252741777 shutdown_connections 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.906+0000 7f8509f22640 1 -- 172.21.15.49:0/252741777 wait complete. 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.907+0000 7f8509f22640 1 Processor -- start 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.907+0000 7f8509f22640 1 -- start start 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8504072a50 0x7f85041ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 0x7f8504158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8504152ea0 0x7f85041e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85040c3f60 con 0x7f85040c7340 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85040c39f0 con 0x7f8504072a50 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8509f22640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f85040c7740 con 0x7f8504152ea0 2024-05-09T08:57:02.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8502ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 0x7f8504158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8502ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 0x7f8504158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49834/0 (socket says 172.21.15.49:49834) 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8502ffd640 1 -- 172.21.15.49:0/137876254 learned_addr learned my addr 172.21.15.49:0/137876254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f8503fff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8504152ea0 0x7f85041e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.908+0000 7f85037fe640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8504072a50 0x7f85041ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 -- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8504152ea0 msgr2=0x7f85041e52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8504152ea0 0x7f85041e52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 -- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8504072a50 msgr2=0x7f85041ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8504072a50 0x7f85041ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85040c8b80 con 0x7f85040c7340 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8503fff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8504152ea0 0x7f85041e52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f85037fe640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8504072a50 0x7f85041ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8502ffd640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 0x7f8504158f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f84f8035040 tx=0x7f84f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.911 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8041030 con 0x7f85040c7340 2024-05-09T08:57:02.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f803d4e0 con 0x7f85040c7340 2024-05-09T08:57:02.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037040 con 0x7f85040c7340 2024-05-09T08:57:02.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85040c6b70 con 0x7f85040c7340 2024-05-09T08:57:02.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.909+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85041f2490 con 0x7f85040c7340 2024-05-09T08:57:02.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.910+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f850404f660 con 0x7f85040c7340 2024-05-09T08:57:02.913 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.911+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84f8044070 con 0x7f85040c7340 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.911+0000 7f84ebfff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84f006be30 0x7f84f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f84f80abc70 con 0x7f85040c7340 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f85037fe640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84f006be30 0x7f84f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f84ebfff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f84f0074210 0x7f84f00766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84f8008eb0 con 0x7f84f0074210 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84f80ac100 con 0x7f85040c7340 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f8503fff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f84f0074210 0x7f84f00766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f85037fe640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84f006be30 0x7f84f006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009560 tx=0x7f84f4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.912+0000 7f8503fff640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f84f0074210 0x7f84f00766b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.913+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84f8008eb0 con 0x7f84f0074210 2024-05-09T08:57:02.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.930+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f85040c30b0 con 0x7f84f0074210 2024-05-09T08:57:02.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.930+0000 7f84ebfff640 1 -- 172.21.15.49:0/137876254 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f85040c30b0 con 0x7f84f0074210 2024-05-09T08:57:02.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f84f0074210 msgr2=0x7f84f00766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f84f0074210 0x7f84f00766b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84f006be30 msgr2=0x7f84f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84f006be30 0x7f84f006e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009560 tx=0x7f84f4009190 comp rx=0 tx=0).stop 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 msgr2=0x7f8504158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 --2- 172.21.15.49:0/137876254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85040c7340 0x7f8504158f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f84f8035040 tx=0x7f84f80046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8503fff640 1 -- 172.21.15.49:0/137876254 reap_dead start 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 shutdown_connections 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 >> 172.21.15.49:0/137876254 conn(0x7f85040c9440 msgr2=0x7f85040c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 shutdown_connections 2024-05-09T08:57:02.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.931+0000 7f8509f22640 1 -- 172.21.15.49:0/137876254 wait complete. 2024-05-09T08:57:02.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-05-09T08:57:02.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.977+0000 7fc3b6aae640 1 Processor -- start 2024-05-09T08:57:02.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.977+0000 7fc3b6aae640 1 -- start start 2024-05-09T08:57:02.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b6aae640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3b00c7130 0x7fc3b0158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b6aae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0072670 0x7fc3b00c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b6aae640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b0159b10 con 0x7fc3b0152e30 2024-05-09T08:57:02.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b6aae640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b0159270 con 0x7fc3b00c7130 2024-05-09T08:57:02.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b6aae640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b0159550 con 0x7fc3b0072670 2024-05-09T08:57:02.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3b00c7130 0x7fc3b0158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fc3b00c7130 0x7fc3b0158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49836/0 (socket says 172.21.15.49:49836) 2024-05-09T08:57:02.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 -- 172.21.15.49:0/2159150356 learned_addr learned my addr 172.21.15.49:0/2159150356 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.978 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b484a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc3b0152e30 0x7fc3b00c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43900/0 (socket says 172.21.15.49:43900) 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3b504b640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0072670 0x7fc3b00c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 -- 172.21.15.49:0/2159150356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0072670 msgr2=0x7fc3b00c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0072670 0x7fc3b00c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 -- 172.21.15.49:0/2159150356 >> v1:172.21.15.49:6789/0 conn(0x7fc3b0152e30 legacy=0x7fc3b00c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.978+0000 7fc3a7fff640 1 -- 172.21.15.49:0/2159150356 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0159830 con 0x7fc3b00c7130 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.979+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b0158be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009870 tx=0x7fc3a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fe2c552c2b5916 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.979+0000 7fc3a77fe640 1 -- 172.21.15.49:0/2159150356 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0037070 con 0x7fc3b00c7130 2024-05-09T08:57:02.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.979+0000 7fc3a77fe640 1 -- 172.21.15.49:0/2159150356 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a0032440 con 0x7fc3b00c7130 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.979+0000 7fc3a77fe640 1 -- 172.21.15.49:0/2159150356 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0032810 con 0x7fc3b00c7130 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 -- 172.21.15.49:0/2159150356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 msgr2=0x7fc3b0158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b0158be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009870 tx=0x7fc3a002a1c0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 -- 172.21.15.49:0/2159150356 shutdown_connections 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0072670 0x7fc3b00c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/2159150356 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b0158be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 -- 172.21.15.49:0/2159150356 >> 172.21.15.49:0/2159150356 conn(0x7fc3b00cc290 msgr2=0x7fc3b00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 -- 172.21.15.49:0/2159150356 shutdown_connections 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.980+0000 7fc3b6aae640 1 -- 172.21.15.49:0/2159150356 wait complete. 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.981+0000 7fc3b6aae640 1 Processor -- start 2024-05-09T08:57:02.981 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.981+0000 7fc3b6aae640 1 -- start start 2024-05-09T08:57:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0072670 0x7fc3b01498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0152e30 0x7fc3b0154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00c6620 con 0x7fc3b00c7130 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b00c59f0 con 0x7fc3b0072670 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b6aae640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b006b4b0 con 0x7fc3b0152e30 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49846/0 (socket says 172.21.15.49:49846) 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 -- 172.21.15.49:0/4035698948 learned_addr learned my addr 172.21.15.49:0/4035698948 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0152e30 0x7fc3b0154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0072670 0x7fc3b01498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0152e30 msgr2=0x7fc3b0154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc3b0152e30 0x7fc3b0154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0072670 msgr2=0x7fc3b01498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:02.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0072670 0x7fc3b01498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.982+0000 7fc3a7fff640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0158c10 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000c00 tx=0x7fc3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0037040 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a0042070 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0030880 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b0157ee0 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.983+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01584a0 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.984+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc3b01529e0 con 0x7fc3b00c7130 2024-05-09T08:57:02.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.984+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3a002ad10 con 0x7fc3b00c7130 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3a57fa640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 0x7fc39c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7fc3a00ab310 con 0x7fc3b00c7130 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 0x7fc39c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v34) ==== 54+0+16 (secure 0 0 0) 0x7fc3b01529e0 con 0x7fc3b00c7130 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc3b014bd80 0x7fc3b0150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.985+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3b0154500 con 0x7fc3b014bd80 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.986+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc3b014bd80 0x7fc3b0150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.986+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 0x7fc39c06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac007fd0 tx=0x7fc3ac0016d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.986+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc3b014bd80 0x7fc3b0150680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:02.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:02.987+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3a8009b50 con 0x7fc3b014bd80 2024-05-09T08:57:02.992 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:02.992 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:02.992 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:02.993+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:03.001 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:03.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.002+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b00c42d0 con 0x7fc3b014bd80 2024-05-09T08:57:03.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.004+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b00c42d0 con 0x7fc3b014bd80 2024-05-09T08:57:03.003 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.004+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc3b006c8a0 0x7fc3b0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.004+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc3b01522a0 con 0x7fc3b006c8a0 2024-05-09T08:57:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.004+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc3b006c8a0 0x7fc3b0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.005+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc3b006c8a0 0x7fc3b0157760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.005 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.005+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b01522a0 con 0x7fc3b006c8a0 2024-05-09T08:57:03.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.021+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b006b990 con 0x7fc3b006c8a0 2024-05-09T08:57:03.021 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.022+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b006b990 con 0x7fc3b006c8a0 2024-05-09T08:57:03.022 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.022+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fc3b01f2cb0 0x7fc3b01f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.022+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc3b01f35d0 con 0x7fc3b01f2cb0 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.023+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fc3b01f2cb0 0x7fc3b01f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.023+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fc3b01f2cb0 0x7fc3b01f3090 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.024+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b01f35d0 con 0x7fc3b01f2cb0 2024-05-09T08:57:03.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.038+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b0158960 con 0x7fc3b01f2cb0 2024-05-09T08:57:03.038 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.040+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b0158960 con 0x7fc3b01f2cb0 2024-05-09T08:57:03.039 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:03.039 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.040+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc3b0062540 0x7fc3b00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.040+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc3b0156830 con 0x7fc3b0062540 2024-05-09T08:57:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.040+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc3b0062540 0x7fc3b00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.040+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc3b0062540 0x7fc3b00ca340 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.040 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.041+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b0156830 con 0x7fc3b0062540 2024-05-09T08:57:03.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.055+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b0156830 con 0x7fc3b0062540 2024-05-09T08:57:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.057+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b0156830 con 0x7fc3b0062540 2024-05-09T08:57:03.056 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:03.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.057+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc3b00cad90 0x7fc3b002c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.057+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc3b00c3470 con 0x7fc3b00cad90 2024-05-09T08:57:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.058+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc3b00cad90 0x7fc3b002c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.058+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc3b00cad90 0x7fc3b002c040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.059+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b00c3470 con 0x7fc3b00cad90 2024-05-09T08:57:03.072 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.073+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b00c3470 con 0x7fc3b00cad90 2024-05-09T08:57:03.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.075+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b00c3470 con 0x7fc3b00cad90 2024-05-09T08:57:03.074 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.075+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc3b002ca90 0x7fc3b002ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.075+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc3b00c8860 con 0x7fc3b002ca90 2024-05-09T08:57:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.075+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc3b002ca90 0x7fc3b002ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.075+0000 7fc3a7fff640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc3b002ca90 0x7fc3b002ceb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.077+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b00c8860 con 0x7fc3b002ca90 2024-05-09T08:57:03.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.084+0000 7f99dc5c1640 1 Processor -- start 2024-05-09T08:57:03.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.084+0000 7f99dc5c1640 1 -- start start 2024-05-09T08:57:03.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dc5c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154b90 0x7f99d40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dc5c1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99d4154f70 0x7f99d40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dc5c1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99d4072600 con 0x7f99d40c6640 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dc5c1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f99d40728e0 con 0x7f99d4154f70 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dc5c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f99d40c5880 con 0x7f99d4154b90 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d9b5c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f99d4154f70 0x7f99d40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dab5e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f99d40c6640 0x7f99d40c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43912/0 (socket says 172.21.15.49:43912) 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99dab5e640 1 -- 172.21.15.49:0/110122476 learned_addr learned my addr 172.21.15.49:0/110122476 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99da35d640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154b90 0x7f99d40cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d9b5c640 1 -- 172.21.15.49:0/110122476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154b90 msgr2=0x7f99d40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d9b5c640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154b90 0x7f99d40cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d9b5c640 1 -- 172.21.15.49:0/110122476 >> v1:172.21.15.49:6789/0 conn(0x7f99d40c6640 legacy=0x7f99d40c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.085 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d9b5c640 1 -- 172.21.15.49:0/110122476 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d40c5b60 con 0x7f99d4154f70 2024-05-09T08:57:03.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.085+0000 7f99d935b640 1 -- 172.21.15.49:0/110122476 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985801622 0 0) 0x7f99d4072600 con 0x7f99d40c6640 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.086+0000 7f99d9b5c640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d4154f70 0x7f99d40cb7c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f99c40097a0 tx=0x7f99c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2af58fa613aac63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.086+0000 7f99d935b640 1 -- 172.21.15.49:0/110122476 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4037070 con 0x7f99d4154f70 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99d935b640 1 -- 172.21.15.49:0/110122476 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c4032440 con 0x7f99d4154f70 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99d935b640 1 -- 172.21.15.49:0/110122476 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4032810 con 0x7f99d4154f70 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 -- 172.21.15.49:0/110122476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d4154f70 msgr2=0x7f99d40cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d4154f70 0x7f99d40cb7c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f99c40097a0 tx=0x7f99c402a210 comp rx=0 tx=0).stop 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 -- 172.21.15.49:0/110122476 shutdown_connections 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d4154f70 0x7f99d40cb7c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/110122476 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154b90 0x7f99d40cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 -- 172.21.15.49:0/110122476 >> 172.21.15.49:0/110122476 conn(0x7f99d4074840 msgr2=0x7f99d4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 -- 172.21.15.49:0/110122476 shutdown_connections 2024-05-09T08:57:03.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.087+0000 7f99dc5c1640 1 -- 172.21.15.49:0/110122476 wait complete. 2024-05-09T08:57:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.088+0000 7f99dc5c1640 1 Processor -- start 2024-05-09T08:57:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.088+0000 7f99dc5c1640 1 -- start start 2024-05-09T08:57:03.090 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 0x7f99d414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d4154b90 0x7f99d41578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154f70 0x7f99d4159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f99d415af40 con 0x7f99d40c6640 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99d40c6120 con 0x7f99d4154b90 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dc5c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f99d4071820 con 0x7f99d4154f70 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 0x7f99d414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 0x7f99d414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49860/0 (socket says 172.21.15.49:49860) 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 -- 172.21.15.49:0/2439596468 learned_addr learned my addr 172.21.15.49:0/2439596468 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99dab5e640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154f70 0x7f99d4159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99d9b5c640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d4154b90 0x7f99d41578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 -- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154f70 msgr2=0x7f99d4159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.091 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99d4154f70 0x7f99d4159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 -- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d4154b90 msgr2=0x7f99d41578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d4154b90 0x7f99d41578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.089+0000 7f99da35d640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d4158310 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99da35d640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 0x7f99d414fdf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e9e0 tx=0x7f99d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0017070 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d000c9d0 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d41561c0 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d000cdc0 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.090+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d4156740 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.091+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99d404f660 con 0x7f99d40c6640 2024-05-09T08:57:03.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.091+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99d001a070 con 0x7f99d40c6640 2024-05-09T08:57:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99caffd640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f99b806be30 0x7f99b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7f99d008b150 con 0x7f99d40c6640 2024-05-09T08:57:03.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99caffd640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f99b8074210 0x7f99b80766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99b8076d80 con 0x7f99b8074210 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99d0092050 con 0x7f99d40c6640 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99d9b5c640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f99b806be30 0x7f99b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.092+0000 7f99dab5e640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f99b8074210 0x7f99b80766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.093+0000 7f99d9b5c640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f99b806be30 0x7f99b806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f99c4034000 tx=0x7f99c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.093+0000 7f99dab5e640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f99b8074210 0x7f99b80766b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.094+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99b8076d80 con 0x7f99b8074210 2024-05-09T08:57:03.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.094+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b00c8860 con 0x7fc3b002ca90 2024-05-09T08:57:03.096 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:03.094+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:03.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.097+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b00c8860 con 0x7fc3b002ca90 2024-05-09T08:57:03.096 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.097+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc3b0206040 0x7fc3b0208430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.097+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc3b00cb280 con 0x7fc3b0206040 2024-05-09T08:57:03.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.097+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc3b0206040 0x7fc3b0208430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.098+0000 7fc3b504b640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc3b0206040 0x7fc3b0208430 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.099+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b00cb280 con 0x7fc3b0206040 2024-05-09T08:57:03.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.111+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f99d415b8a0 con 0x7f99b8074210 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.111+0000 7f99caffd640 1 -- 172.21.15.49:0/2439596468 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f99d415b8a0 con 0x7f99b8074210 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.111+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f99b8074210 msgr2=0x7f99b80766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.111+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f99b8074210 0x7f99b80766b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f99b806be30 msgr2=0x7f99b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f99b806be30 0x7f99b806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f99c4034000 tx=0x7f99c402e000 comp rx=0 tx=0).stop 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 msgr2=0x7f99d414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 --2- 172.21.15.49:0/2439596468 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f99d40c6640 0x7f99d414fdf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e9e0 tx=0x7f99d000eea0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.111 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dab5e640 1 -- 172.21.15.49:0/2439596468 reap_dead start 2024-05-09T08:57:03.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 shutdown_connections 2024-05-09T08:57:03.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 >> 172.21.15.49:0/2439596468 conn(0x7f99d4074840 msgr2=0x7f99d4152a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 shutdown_connections 2024-05-09T08:57:03.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.112+0000 7f99dc5c1640 1 -- 172.21.15.49:0/2439596468 wait complete. 2024-05-09T08:57:03.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.113+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b002d3f0 con 0x7fc3b0206040 2024-05-09T08:57:03.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.116+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b002d3f0 con 0x7fc3b0206040 2024-05-09T08:57:03.115 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:03.115 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:03.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.116+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc3b0208e80 0x7fc3b020b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.116+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc3b020e040 con 0x7fc3b0208e80 2024-05-09T08:57:03.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.116+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc3b0208e80 0x7fc3b020b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.117+0000 7fc3b484a640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc3b0208e80 0x7fc3b020b270 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.117 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.118+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3b020e040 con 0x7fc3b0208e80 2024-05-09T08:57:03.121 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-05-09T08:57:03.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.133+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc3b020b7b0 con 0x7fc3b0208e80 2024-05-09T08:57:03.134 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:03.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.134+0000 7fc3a57fa640 1 -- 172.21.15.49:0/4035698948 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3b020b7b0 con 0x7fc3b0208e80 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc3b014bd80 msgr2=0x7fc3b0150680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc3b014bd80 0x7fc3b0150680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc3b006c8a0 msgr2=0x7fc3b0157760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc3b006c8a0 0x7fc3b0157760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fc3b01f2cb0 msgr2=0x7fc3b01f3090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fc3b01f2cb0 0x7fc3b01f3090 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc3b0062540 msgr2=0x7fc3b00ca340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc3b0062540 0x7fc3b00ca340 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc3b00cad90 msgr2=0x7fc3b002c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc3b00cad90 0x7fc3b002c040 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b504b640 1 -- 172.21.15.49:0/4035698948 reap_dead start 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc3b002ca90 msgr2=0x7fc3b002ceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc3b002ca90 0x7fc3b002ceb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b504b640 1 -- 172.21.15.49:0/4035698948 reap_dead start 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc3b0206040 msgr2=0x7fc3b0208430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc3b0206040 0x7fc3b0208430 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc3b0208e80 msgr2=0x7fc3b020b270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc3b0208e80 0x7fc3b020b270 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b504b640 1 -- 172.21.15.49:0/4035698948 reap_dead start 2024-05-09T08:57:03.137 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 msgr2=0x7fc39c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 0x7fc39c06e2f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac007fd0 tx=0x7fc3ac0016d0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 msgr2=0x7fc3b01efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0000c00 tx=0x7fc3a00046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 shutdown_connections 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc39c06be30 0x7fc39c06e2f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 --2- 172.21.15.49:0/4035698948 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc3b00c7130 0x7fc3b01efcb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 >> 172.21.15.49:0/4035698948 conn(0x7fc3b00cc290 msgr2=0x7fc3b006da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 shutdown_connections 2024-05-09T08:57:03.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.135+0000 7fc3b6aae640 1 -- 172.21.15.49:0/4035698948 wait complete. 2024-05-09T08:57:03.148 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:03.148 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:03.148 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:03.148 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:03.148 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:03.157 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:03.194 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:03.195+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:03.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.237+0000 7fa430227640 1 Processor -- start 2024-05-09T08:57:03.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.237+0000 7fa430227640 1 -- start start 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa430227640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4281512d0 0x7fa42806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa430227640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806b9f0 0x7fa42806bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa430227640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa428158000 con 0x7fa42806c4c0 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa430227640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fa428157480 con 0x7fa4281512d0 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa430227640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa428157760 con 0x7fa42806b9f0 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4281512d0 0x7fa42806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42e7c4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fa42806c4c0 0x7fa42815ac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43918/0 (socket says 172.21.15.49:43918) 2024-05-09T08:57:03.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fa4281512d0 0x7fa42806b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49876/0 (socket says 172.21.15.49:49876) 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42e7c4640 1 -- 172.21.15.49:0/1856637656 learned_addr learned my addr 172.21.15.49:0/1856637656 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42d7c2640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806b9f0 0x7fa42806bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1856637656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806b9f0 msgr2=0x7fa42806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806b9f0 0x7fa42806bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1856637656 >> v1:172.21.15.49:6789/0 conn(0x7fa42806c4c0 legacy=0x7fa42815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42cfc1640 1 -- 172.21.15.49:0/1856637656 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1087113856 0 0) 0x7fa428158000 con 0x7fa42806c4c0 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.238+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1856637656 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa428157a40 con 0x7fa4281512d0 2024-05-09T08:57:03.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.239+0000 7fa42dfc3640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4281512d0 0x7fa42806b4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa42402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43255e0387ea42fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.239+0000 7fa42cfc1640 1 -- 172.21.15.49:0/1856637656 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424037070 con 0x7fa4281512d0 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.239+0000 7fa42cfc1640 1 -- 172.21.15.49:0/1856637656 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa424032440 con 0x7fa4281512d0 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.239+0000 7fa42cfc1640 1 -- 172.21.15.49:0/1856637656 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424032810 con 0x7fa4281512d0 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 -- 172.21.15.49:0/1856637656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4281512d0 msgr2=0x7fa42806b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4281512d0 0x7fa42806b4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa42402a210 comp rx=0 tx=0).stop 2024-05-09T08:57:03.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 -- 172.21.15.49:0/1856637656 shutdown_connections 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806b9f0 0x7fa42806bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 --2- 172.21.15.49:0/1856637656 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa4281512d0 0x7fa42806b4b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 -- 172.21.15.49:0/1856637656 >> 172.21.15.49:0/1856637656 conn(0x7fa4280cc290 msgr2=0x7fa4280cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 -- 172.21.15.49:0/1856637656 shutdown_connections 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.240+0000 7fa430227640 1 -- 172.21.15.49:0/1856637656 wait complete. 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.241+0000 7fa430227640 1 Processor -- start 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.241+0000 7fa430227640 1 -- start start 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 0x7fa4281e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806c4c0 0x7fa4280cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4281512d0 0x7fa4280cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281570b0 con 0x7fa42806b9f0 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4280c8af0 con 0x7fa4281512d0 2024-05-09T08:57:03.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa430227640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa428157230 con 0x7fa42806c4c0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 0x7fa4281e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 0x7fa4281e3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49878/0 (socket says 172.21.15.49:49878) 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1514530126 learned_addr learned my addr 172.21.15.49:0/1514530126 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42d7c2640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806c4c0 0x7fa4280cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42e7c4640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4281512d0 0x7fa4280cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806c4c0 msgr2=0x7fa4280cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa42806c4c0 0x7fa4280cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4281512d0 msgr2=0x7fa4280cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4281512d0 0x7fa4280cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa428149560 con 0x7fa42806b9f0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa42dfc3640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 0x7fa4281e3ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa4240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424037050 con 0x7fa42806b9f0 2024-05-09T08:57:03.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.242+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa424038040 con 0x7fa42806b9f0 2024-05-09T08:57:03.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.243+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281f3780 con 0x7fa42806b9f0 2024-05-09T08:57:03.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.243+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424004490 con 0x7fa42806b9f0 2024-05-09T08:57:03.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.243+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281f3d40 con 0x7fa42806b9f0 2024-05-09T08:57:03.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.243+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa42804f660 con 0x7fa42806b9f0 2024-05-09T08:57:03.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.244+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa42403c6d0 con 0x7fa42806b9f0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.244+0000 7fa41e7fc640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5857+0+0 (secure 0 0 0) 0x7fa4240ab8a0 con 0x7fa42806b9f0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa42d7c2640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa40c06be30 0x7fa40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa41e7fc640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa40c0741b0 0x7fa40c076650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa42400af30 con 0x7fa40c0741b0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa4240b02c0 con 0x7fa42806b9f0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa42e7c4640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa40c0741b0 0x7fa40c076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa42d7c2640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c64f0 tx=0x7fa41800a2a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.245+0000 7fa42e7c4640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa40c0741b0 0x7fa40c076650 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.246+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa42400af30 con 0x7fa40c0741b0 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.537075942999998, 2024-05-09T08:57:03.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:03.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.537055344000002, 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:03.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:03.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:03.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.537050473000001, 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.278 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:03.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.533492174000003, 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:03.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:03.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.291 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.533483945999997, 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:03.292 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:03.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:03.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:03.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:03.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.533480136999998, 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.300 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:03.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:03.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.530924108999997, 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.306 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:03.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:03.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.516423302, 2024-05-09T08:57:03.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:03.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:03.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.516407831000002, 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:03.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:03.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.516403087, 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:03.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:03.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.516396669999999, 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:03.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:03.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.340 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500420450999997, 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:03.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:03.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:03.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:03.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500407748999997, 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.353 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500399831999999, 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:03.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:03.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:03.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500397323999998, 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:03.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:03.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.50039486, 2024-05-09T08:57:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:03.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500379565000003, 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:03.371 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:03.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:03.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500375243999997, 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:03.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:03.381 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.500372104, 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:03.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.385 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.497912304000003, 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:03.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:03.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:03.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:03.390 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:03.391 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:03.393 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.263+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fa42806cac0 con 0x7fa40c0741b0 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa41e7fc640 1 -- 172.21.15.49:0/1514530126 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fa42806cac0 con 0x7fa40c0741b0 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa40c0741b0 msgr2=0x7fa40c076650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fa40c0741b0 0x7fa40c076650 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa40c06be30 msgr2=0x7fa40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa40c06be30 0x7fa40c06e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4280c64f0 tx=0x7fa41800a2a0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 msgr2=0x7fa4281e3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 --2- 172.21.15.49:0/1514530126 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa42806b9f0 0x7fa4281e3ff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097a0 tx=0x7fa4240046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa42e7c4640 1 -- 172.21.15.49:0/1514530126 reap_dead start 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 shutdown_connections 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 >> 172.21.15.49:0/1514530126 conn(0x7fa4280cc290 msgr2=0x7fa4281564d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 shutdown_connections 2024-05-09T08:57:03.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.264+0000 7fa430227640 1 -- 172.21.15.49:0/1514530126 wait complete. 2024-05-09T08:57:03.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-05-09T08:57:03.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.359+0000 7f467dffb640 1 -- 172.21.15.49:0/2399931212 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v35) ==== 101+0+0 (secure 0 0 0) 0x7f4670079fd0 con 0x7f4680154f70 2024-05-09T08:57:03.397 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:57:03.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 msgr2=0x7f466c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 0x7f466c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4678009ba0 tx=0x7f4678008040 comp rx=0 tx=0).stop 2024-05-09T08:57:03.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 msgr2=0x7f46800b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4670009d90 tx=0x7f4670004a40 comp rx=0 tx=0).stop 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 shutdown_connections 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f466c06be30 0x7f466c06e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4680154f70 0x7f46800b58a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4680154b90 0x7f468014c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 --2- 172.21.15.49:0/2399931212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46800c6d50 0x7f468014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.361+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 >> 172.21.15.49:0/2399931212 conn(0x7f4680074840 msgr2=0x7f4680153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.362+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 shutdown_connections 2024-05-09T08:57:03.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.362+0000 7f468769a640 1 -- 172.21.15.49:0/2399931212 wait complete. 2024-05-09T08:57:03.398 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:03.393+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:03.493 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:03.494+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:03.499 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:03.499 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:03.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:03.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.539+0000 7f84efcbd640 1 Processor -- start 2024-05-09T08:57:03.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.540+0000 7f84efcbd640 1 -- start start 2024-05-09T08:57:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.540+0000 7f84efcbd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84e80c9ec0 0x7f84e815b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84efcbd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e806cdc0 con 0x7f84e80c71a0 2024-05-09T08:57:03.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84efcbd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e806d080 con 0x7f84e8155090 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84efcbd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f84e80c8a20 con 0x7f84e80c9ec0 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ee25a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f84e80c9ec0 0x7f84e815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ed258640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f84e80c71a0 0x7f84e815ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43930/0 (socket says 172.21.15.49:43930) 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ed258640 1 -- 172.21.15.49:0/2633424780 learned_addr learned my addr 172.21.15.49:0/2633424780 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ee25a640 1 -- 172.21.15.49:0/2633424780 >> v1:172.21.15.183:6789/0 conn(0x7f84e8155090 legacy=0x7f84e80bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ee25a640 1 -- 172.21.15.49:0/2633424780 >> v1:172.21.15.49:6789/0 conn(0x7f84e80c71a0 legacy=0x7f84e815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84ee25a640 1 -- 172.21.15.49:0/2633424780 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e806d360 con 0x7f84e80c9ec0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.541+0000 7f84eca57640 1 -- 172.21.15.49:0/2633424780 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848121663 0 0) 0x7f84e806cdc0 con 0x7f84e80c71a0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84ee25a640 1 --2- 172.21.15.49:0/2633424780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c9ec0 0x7f84e815b560 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0307bb49d54d7c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84eca57640 1 -- 172.21.15.49:0/2633424780 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0037070 con 0x7f84e80c9ec0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84eca57640 1 -- 172.21.15.49:0/2633424780 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0007740 con 0x7f84e80c9ec0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84eca57640 1 -- 172.21.15.49:0/2633424780 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0007b10 con 0x7f84e80c9ec0 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84efcbd640 1 -- 172.21.15.49:0/2633424780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c9ec0 msgr2=0x7f84e815b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84efcbd640 1 --2- 172.21.15.49:0/2633424780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c9ec0 0x7f84e815b560 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e002a780 comp rx=0 tx=0).stop 2024-05-09T08:57:03.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84efcbd640 1 -- 172.21.15.49:0/2633424780 shutdown_connections 2024-05-09T08:57:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84efcbd640 1 --2- 172.21.15.49:0/2633424780 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c9ec0 0x7f84e815b560 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.542+0000 7f84efcbd640 1 -- 172.21.15.49:0/2633424780 >> 172.21.15.49:0/2633424780 conn(0x7f84e80c93a0 msgr2=0x7f84e80c97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.543+0000 7f84efcbd640 1 -- 172.21.15.49:0/2633424780 shutdown_connections 2024-05-09T08:57:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.543+0000 7f84efcbd640 1 -- 172.21.15.49:0/2633424780 wait complete. 2024-05-09T08:57:03.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 Processor -- start 2024-05-09T08:57:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 -- start start 2024-05-09T08:57:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 0x7f84e81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e80c9ec0 0x7f84e8158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84e8155090 0x7f84e8071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80bec90 con 0x7f84e80c71a0 2024-05-09T08:57:03.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80be9b0 con 0x7f84e8155090 2024-05-09T08:57:03.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.544+0000 7f84efcbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80bef70 con 0x7f84e80c9ec0 2024-05-09T08:57:03.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 0x7f84e81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 0x7f84e81587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49896/0 (socket says 172.21.15.49:49896) 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 -- 172.21.15.49:0/3988035736 learned_addr learned my addr 172.21.15.49:0/3988035736 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84ed258640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e80c9ec0 0x7f84e8158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 -- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e80c9ec0 msgr2=0x7f84e8158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84e80c9ec0 0x7f84e8158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 -- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84e8155090 msgr2=0x7f84e8071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84e8155090 0x7f84e8071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e81f16b0 con 0x7f84e80c71a0 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84eda59640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 0x7f84e81587c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c430 tx=0x7f84e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4017070 con 0x7f84e80c71a0 2024-05-09T08:57:03.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e40052d0 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81f4060 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4003830 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.545+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81f4620 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.546+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84e81f3e10 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84e401a070 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84cc06be30 0x7f84cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5857+0+0 (secure 0 0 0) 0x7f84e408a1f0 con 0x7f84e80c71a0 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f84cc074210 0x7f84cc0766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84cc076d80 con 0x7f84cc074210 2024-05-09T08:57:03.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f84e4091050 con 0x7f84e80c71a0 2024-05-09T08:57:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.547+0000 7f84ed258640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84cc06be30 0x7f84cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.548+0000 7f84ee25a640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f84cc074210 0x7f84cc0766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.548+0000 7f84ed258640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84cc06be30 0x7f84cc06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e80718e0 tx=0x7f84dc007f00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.548+0000 7f84ee25a640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f84cc074210 0x7f84cc0766b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:03.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.549+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84cc076d80 con 0x7f84cc074210 2024-05-09T08:57:03.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f84e804f660 con 0x7f84cc074210 2024-05-09T08:57:03.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84da7fc640 1 -- 172.21.15.49:0/3988035736 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f84e804f660 con 0x7f84cc074210 2024-05-09T08:57:03.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f84cc074210 msgr2=0x7f84cc0766b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f84cc074210 0x7f84cc0766b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84cc06be30 msgr2=0x7f84cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f84cc06be30 0x7f84cc06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f84e80718e0 tx=0x7f84dc007f00 comp rx=0 tx=0).stop 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 msgr2=0x7f84e81587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 --2- 172.21.15.49:0/3988035736 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f84e80c71a0 0x7f84e81587c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c430 tx=0x7f84e400c8f0 comp rx=0 tx=0).stop 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84ee25a640 1 -- 172.21.15.49:0/3988035736 reap_dead start 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 shutdown_connections 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 >> 172.21.15.49:0/3988035736 conn(0x7f84e80c93a0 msgr2=0x7f84e8154130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.566+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 shutdown_connections 2024-05-09T08:57:03.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:03.567+0000 7f84efcbd640 1 -- 172.21.15.49:0/3988035736 wait complete. 2024-05-09T08:57:03.618 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:03.618+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:04.082 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:04.083 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:04.083 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:04.084 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:04.080+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:04.180 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:04.181+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:04.230 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:04.231 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.549225194000002, 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:04.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:04.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:04.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.242 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.549222327000003, 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:04.243 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:04.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:04.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:04.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.549218682000003, 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.255 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:04.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:04.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:04.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.262 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.506409517000002, 2024-05-09T08:57:04.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:04.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:04.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.501617527999997, 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.271 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:04.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.276 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.499111159999998, 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:04.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:04.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:04.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.497639769999999, 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.281 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:04.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:04.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.288 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 46.497563931999998, 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:04.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484742521999998, 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:04.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:04.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484721524000001, 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.299 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:04.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.48471713, 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:04.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484712072000001, 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:04.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.316 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484708591, 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:04.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:04.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484703973000002, 2024-05-09T08:57:04.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:04.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484700504000003, 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.328 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:04.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:04.333 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484697203000003, 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:04.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:04.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:04.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:04.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.484384134000003, 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:04.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.468308311999998, 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:04.344 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:04.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.349 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.468303632999998, 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:04.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.355 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.468297614999997, 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:04.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.361 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:04.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:04.363 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:04.364+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:04.464 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:04.465+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:04.509 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:04.509 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:04.517 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:04.565 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:04.566+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:04.620 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:04.620 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:04.620 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:04.620 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:04.620 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:04.629 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:04.670 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:04.671+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:04.767 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:04.768+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:04.774 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:04.774 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.110846413000004, 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.775 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:04.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:04.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:04.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.718190148999994, 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.782 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:04.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:04.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:04.788 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.718142556999993, 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:04.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:04.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.718092690000006, 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.795 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.718086889999995, 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717975089999996, 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717877928000007, 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717872170000007, 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717798529999996, 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717746718000001, 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.717742998999995, 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.232686541999996, 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.232670057999997, 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.232588101000005, 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.232540541000006, 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.863 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.028912097000003, 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:04.869 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:04.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:04.875 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.012832652, 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:04.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:04.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.012828335999998, 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:04.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.012819372999999, 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.887 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 35.012815312999997, 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:04.893 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:04.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:04.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:04.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.898 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:04.899 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:04.899 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:04.900 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:04.901 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:04.901+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:05.001 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:05.002+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:05.024 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:05.024 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:05.024 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:05.024 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:05.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:05.104 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:05.105+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:05.145 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:05.146 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:05.146 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:05.147 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:05.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:05.240 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:05.241+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.249+0000 7f1df80bc640 1 Processor -- start 2024-05-09T08:57:05.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.249+0000 7f1df80bc640 1 -- start start 2024-05-09T08:57:05.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df80bc640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1df0154b90 0x7f1df00cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df80bc640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 0x7f1df00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df80bc640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df0072600 con 0x7f1df00c6640 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df80bc640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1df00728e0 con 0x7f1df0154b90 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df80bc640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1df00c5880 con 0x7f1df0154f70 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1df0154b90 0x7f1df00cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1df0154b90 0x7f1df00cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49902/0 (socket says 172.21.15.49:49902) 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 -- 172.21.15.49:0/3127735678 learned_addr learned my addr 172.21.15.49:0/3127735678 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 -- 172.21.15.49:0/3127735678 >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 msgr2=0x7f1df00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5657640 1 --2- 172.21.15.49:0/3127735678 >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 0x7f1df00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3127735678 >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 0x7f1df00cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 -- 172.21.15.49:0/3127735678 >> v1:172.21.15.183:6789/0 conn(0x7f1df00c6640 legacy=0x7f1df00c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5e58640 1 -- 172.21.15.49:0/3127735678 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df00c5b60 con 0x7f1df0154b90 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.250+0000 7f1df5657640 1 --2- 172.21.15.49:0/3127735678 >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 0x7f1df00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.251+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3127735678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df0154b90 0x7f1df00cb280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df00cdd20 tx=0x7f1dec02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31dd6616b575623 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.251+0000 7f1df4e56640 1 -- 172.21.15.49:0/3127735678 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec036070 con 0x7f1df0154b90 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.251+0000 7f1df4e56640 1 -- 172.21.15.49:0/3127735678 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec031600 con 0x7f1df0154b90 2024-05-09T08:57:05.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.251+0000 7f1df4e56640 1 -- 172.21.15.49:0/3127735678 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0319d0 con 0x7f1df0154b90 2024-05-09T08:57:05.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 -- 172.21.15.49:0/3127735678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df0154b90 msgr2=0x7f1df00cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3127735678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df0154b90 0x7f1df00cb280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df00cdd20 tx=0x7f1dec02a780 comp rx=0 tx=0).stop 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 -- 172.21.15.49:0/3127735678 shutdown_connections 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3127735678 >> v2:172.21.15.183:3300/0 conn(0x7f1df0154f70 0x7f1df00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3127735678 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df0154b90 0x7f1df00cb280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 -- 172.21.15.49:0/3127735678 >> 172.21.15.49:0/3127735678 conn(0x7f1df0074840 msgr2=0x7f1df0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 -- 172.21.15.49:0/3127735678 shutdown_connections 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.252+0000 7f1df80bc640 1 -- 172.21.15.49:0/3127735678 wait complete. 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.253+0000 7f1df80bc640 1 Processor -- start 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.253+0000 7f1df80bc640 1 -- start start 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1df0154b90 0x7f1df0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df0154f70 0x7f1df00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1df015af40 con 0x7f1df00c6640 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df00c6120 con 0x7f1df0154f70 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df80bc640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1df0071820 con 0x7f1df0154b90 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df5e58640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df5e58640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49908/0 (socket says 172.21.15.49:49908) 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df5e58640 1 -- 172.21.15.49:0/3112293804 learned_addr learned my addr 172.21.15.49:0/3112293804 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1df0154b90 0x7f1df0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.254+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df0154f70 0x7f1df00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1df0154b90 msgr2=0x7f1df0157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1df0154b90 0x7f1df0157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df0154f70 msgr2=0x7f1df00cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df0154f70 0x7f1df00cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df01580d0 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df0154f70 0x7f1df00cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034040 tx=0x7f1dec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec036050 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec042070 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df01583b0 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec009510 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.255+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df0156450 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.256+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1df004f660 con 0x7f1df00c6640 2024-05-09T08:57:05.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.256+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1dec007c50 con 0x7f1df00c6640 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1de67fc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 0x7f1dd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5857+0+0 (secure 0 0 0) 0x7f1dec0abec0 con 0x7f1df00c6640 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7f1dec0ae540 con 0x7f1df00c6640 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 0x7f1dd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1df00d0d90 0x7f1df00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.257+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1df0157e80 con 0x7f1df00d0d90 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.258+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1df00d0d90 0x7f1df00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.258+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 0x7f1dd406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1df00b6140 tx=0x7f1de0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.258+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1df00d0d90 0x7f1df00cbde0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.259+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df0157e80 con 0x7f1df00d0d90 2024-05-09T08:57:05.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.274+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df0158920 con 0x7f1df00d0d90 2024-05-09T08:57:05.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.275+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df0158920 con 0x7f1df00d0d90 2024-05-09T08:57:05.275 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:05.275 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:05.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.276+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1df00cc320 0x7f1df01f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.276+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1df0075060 con 0x7f1df00cc320 2024-05-09T08:57:05.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.276+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1df00cc320 0x7f1df01f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.276+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1df00cc320 0x7f1df01f0f60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.277+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df0075060 con 0x7f1df00cc320 2024-05-09T08:57:05.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.292+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df01f16f0 con 0x7f1df00cc320 2024-05-09T08:57:05.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.293+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df01f16f0 con 0x7f1df00cc320 2024-05-09T08:57:05.294 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:05.294 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:05.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.294+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1df01f4890 0x7f1df0061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.294+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1df01f1900 con 0x7f1df01f4890 2024-05-09T08:57:05.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.294+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1df01f4890 0x7f1df0061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.294+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1df01f4890 0x7f1df0061c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.295+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df01f1900 con 0x7f1df01f4890 2024-05-09T08:57:05.305 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:05.306+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:05.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.309+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df00621b0 con 0x7f1df01f4890 2024-05-09T08:57:05.309 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:05.309 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:05.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.311+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df00621b0 con 0x7f1df01f4890 2024-05-09T08:57:05.310 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:05.311 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:05.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.311+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1df00b4660 0x7f1df00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.311+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1df00621b0 con 0x7f1df00b4660 2024-05-09T08:57:05.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.311+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1df00b4660 0x7f1df00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.311+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1df00b4660 0x7f1df00b4a40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.312+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df00621b0 con 0x7f1df00b4660 2024-05-09T08:57:05.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:05.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.327+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df00621b0 con 0x7f1df00b4660 2024-05-09T08:57:05.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.329+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df00621b0 con 0x7f1df00b4660 2024-05-09T08:57:05.328 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.329+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1df002c040 0x7f1df002c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.329+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1df01f4ee0 con 0x7f1df002c040 2024-05-09T08:57:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.329+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1df002c040 0x7f1df002c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.330+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1df002c040 0x7f1df002c420 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.331+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df01f4ee0 con 0x7f1df002c040 2024-05-09T08:57:05.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.346+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df01f4ee0 con 0x7f1df002c040 2024-05-09T08:57:05.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.347+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df01f4ee0 con 0x7f1df002c040 2024-05-09T08:57:05.347 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:05.347 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:05.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.348+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1df002ce70 0x7f1df0207350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.348+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1df002d2d0 con 0x7f1df002ce70 2024-05-09T08:57:05.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.348+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1df002ce70 0x7f1df0207350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.348+0000 7f1df5657640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1df002ce70 0x7f1df0207350 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.349+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df002d2d0 con 0x7f1df002ce70 2024-05-09T08:57:05.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.364+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df002d2d0 con 0x7f1df002ce70 2024-05-09T08:57:05.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.366+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df002d2d0 con 0x7f1df002ce70 2024-05-09T08:57:05.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.366+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 0x7f1df020a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.366+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1df020cf60 con 0x7f1df0207da0 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.366+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 0x7f1df020a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.367+0000 7f1df6659640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 0x7f1df020a190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.367+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df020cf60 con 0x7f1df0207da0 2024-05-09T08:57:05.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.382+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df020a6d0 con 0x7f1df0207da0 2024-05-09T08:57:05.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.384+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df020a6d0 con 0x7f1df0207da0 2024-05-09T08:57:05.383 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.384+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 0x7f1df020d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.384+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1df0210110 con 0x7f1df020ac30 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.384+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 0x7f1df020d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.384+0000 7f1df5e58640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 0x7f1df020d0a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.385+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1df0210110 con 0x7f1df020ac30 2024-05-09T08:57:05.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.399+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1df02100e0 con 0x7f1df020ac30 2024-05-09T08:57:05.399 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:57:05.401 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.401+0000 7f1de67fc640 1 -- 172.21.15.49:0/3112293804 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1df02100e0 con 0x7f1df020ac30 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1df00d0d90 msgr2=0x7f1df00cbde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f1df00d0d90 0x7f1df00cbde0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1df00cc320 msgr2=0x7f1df01f0f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1df00cc320 0x7f1df01f0f60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1df01f4890 msgr2=0x7f1df0061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f1df01f4890 0x7f1df0061c70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1df00b4660 msgr2=0x7f1df00b4a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f1df00b4660 0x7f1df00b4a40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.403 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1df002c040 msgr2=0x7f1df002c420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1df002c040 0x7f1df002c420 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1df002ce70 msgr2=0x7f1df0207350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f1df002ce70 0x7f1df0207350 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 msgr2=0x7f1df020a190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 0x7f1df020a190 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df6659640 1 -- 172.21.15.49:0/3112293804 reap_dead start 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df6659640 1 -- 172.21.15.49:0/3112293804 reap_dead start 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df6659640 1 -- 172.21.15.49:0/3112293804 reap_dead start 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 msgr2=0x7f1df020d0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 0x7f1df020d0a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 msgr2=0x7f1dd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 0x7f1dd406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1df00b6140 tx=0x7f1de0008040 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 msgr2=0x7f1df00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034040 tx=0x7f1dec004a40 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 shutdown_connections 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1df020ac30 0x7f1df020d0a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f1df0207da0 0x7f1df020a190 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1dd406be30 0x7f1dd406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 --2- 172.21.15.49:0/3112293804 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1df00c6640 0x7f1df00d0370 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 >> 172.21.15.49:0/3112293804 conn(0x7f1df0074840 msgr2=0x7f1df00ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 shutdown_connections 2024-05-09T08:57:05.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.402+0000 7f1df80bc640 1 -- 172.21.15.49:0/3112293804 wait complete. 2024-05-09T08:57:05.406 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:05.408+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:05.439 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:05.439 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:05.448 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:05.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.507+0000 7f8d78316640 1 Processor -- start 2024-05-09T08:57:05.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.507+0000 7f8d78316640 1 -- start start 2024-05-09T08:57:05.508 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:05.508+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d78316640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8d700cd280 0x7f8d700c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d78316640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d700cdc50 con 0x7f8d70151c00 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d78316640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d70157ac0 con 0x7f8d700cd710 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d78316640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8d70157da0 con 0x7f8d700cd280 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d760b2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8d70151c00 0x7f8d700c96a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43944/0 (socket says 172.21.15.49:43944) 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d760b2640 1 -- 172.21.15.49:0/3995839035 learned_addr learned my addr 172.21.15.49:0/3995839035 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:05.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d758b1640 1 --2- 172.21.15.49:0/3995839035 >> v2:172.21.15.183:3300/0 conn(0x7f8d700cd280 0x7f8d700c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600278416 0 0) 0x7f8d700cdc50 con 0x7f8d70151c00 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.508+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d5c003520 con 0x7f8d70151c00 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448113169 0 0) 0x7f8d70157ac0 con 0x7f8d700cd710 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d700cdc50 con 0x7f8d700cd710 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1047768148 0 0) 0x7f8d5c003520 con 0x7f8d70151c00 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 >> v2:172.21.15.183:3300/0 conn(0x7f8d700cd280 msgr2=0x7f8d700c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 --2- 172.21.15.49:0/3995839035 >> v2:172.21.15.183:3300/0 conn(0x7f8d700cd280 0x7f8d700c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 >> v1:172.21.15.183:6789/0 conn(0x7f8d700cd710 legacy=0x7f8d7015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d70158080 con 0x7f8d70151c00 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f8d60001670 con 0x7f8d70151c00 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d758b1640 1 --2- 172.21.15.49:0/3995839035 >> v2:172.21.15.183:3300/0 conn(0x7f8d700cd280 0x7f8d700c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:05.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 >> v1:172.21.15.49:6789/0 conn(0x7f8d70151c00 legacy=0x7f8d700c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 0x7f8d5c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70157da0 con 0x7f8d5c003b40 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c00bfb0 con 0x7f8d5c004800 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c00c1b0 con 0x7f8d5c0051c0 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 0x7f8d5c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d758b1640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d768b3640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 -- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0051c0 msgr2=0x7f8d5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0051c0 0x7f8d5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 -- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d5c004800 msgr2=0x7f8d5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d5c004800 0x7f8d5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d760b2640 1 -- 172.21.15.49:0/3995839035 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c003520 con 0x7f8d5c003b40 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.509+0000 7f8d768b3640 1 -- 172.21.15.49:0/3995839035 reap_dead start 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.510+0000 7f8d760b2640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 0x7f8d5c003f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007c80 tx=0x7f8d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48f6c8d560533ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.510+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60038070 con 0x7f8d5c003b40 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.510+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60005af0 con 0x7f8d5c003b40 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.510+0000 7f8d750b0640 1 -- 172.21.15.49:0/3995839035 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60007cc0 con 0x7f8d5c003b40 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 -- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 msgr2=0x7f8d5c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 0x7f8d5c003f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007c80 tx=0x7f8d5c005830 comp rx=0 tx=0).stop 2024-05-09T08:57:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 -- 172.21.15.49:0/3995839035 shutdown_connections 2024-05-09T08:57:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 --2- 172.21.15.49:0/3995839035 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c003b40 0x7f8d5c003f40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 -- 172.21.15.49:0/3995839035 >> 172.21.15.49:0/3995839035 conn(0x7f8d7006b200 msgr2=0x7f8d7006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 -- 172.21.15.49:0/3995839035 shutdown_connections 2024-05-09T08:57:05.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.511+0000 7f8d78316640 1 -- 172.21.15.49:0/3995839035 wait complete. 2024-05-09T08:57:05.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.512+0000 7f8d78316640 1 Processor -- start 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.512+0000 7f8d78316640 1 -- start start 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c003b40 0x7f8d7014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d70146f70 0x7f8d70150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70151fe0 con 0x7f8d701508a0 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70158640 con 0x7f8d70146f70 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d78316640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8d700c96a0 con 0x7f8d5c003b40 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49922/0 (socket says 172.21.15.49:49922) 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 -- 172.21.15.49:0/2462274312 learned_addr learned my addr 172.21.15.49:0/2462274312 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d760b2640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c003b40 0x7f8d7014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d758b1640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d70146f70 0x7f8d70150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 -- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c003b40 msgr2=0x7f8d7014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c003b40 0x7f8d7014f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 -- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d70146f70 msgr2=0x7f8d70150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d70146f70 0x7f8d70150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.513+0000 7f8d768b3640 1 -- 172.21.15.49:0/2462274312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d701511e0 con 0x7f8d701508a0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d768b3640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d701579b0 tx=0x7f8d6800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68019070 con 0x7f8d701508a0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d68006e20 con 0x7f8d701508a0 2024-05-09T08:57:05.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7014dec0 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68003b70 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.514+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d700ccc10 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.515+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d70157ac0 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.515+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d6801c070 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.516+0000 7f8d66ffd640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 0x7f8d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.516+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5857+0+0 (secure 0 0 0) 0x7f8d6808a1a0 con 0x7f8d701508a0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.516+0000 7f8d760b2640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 0x7f8d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:05.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.517+0000 7f8d760b2640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 0x7f8d5406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d60036040 tx=0x7f8d60009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:05.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.521+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d70157ac0 con 0x7f8d701508a0 2024-05-09T08:57:05.599 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:05.600 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:05.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:05.609 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:05.610+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:05.709 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:05.710+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:05.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:05.716+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f8d7004f660 con 0x7f8d701508a0 2024-05-09T08:57:05.761 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:05.761 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:05.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:57:05.812 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:05.813+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:05.911 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:05.912+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:05.922 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:05.922 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:05.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:57:06.012 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:06.013+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:06.040 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.041 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:57:06.114 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:06.115+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:06.159 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.159 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:06.215 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:06.216+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:06.316 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:06.317+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:06.318 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.318 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:06.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.369+0000 7f8d66ffd640 1 -- 172.21.15.49:0/2462274312 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v36) ==== 87+0+0 (secure 0 0 0) 0x7f8d68058b40 con 0x7f8d701508a0 2024-05-09T08:57:06.368 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 msgr2=0x7f8d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 0x7f8d5406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d60036040 tx=0x7f8d60009030 comp rx=0 tx=0).stop 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 msgr2=0x7f8d701eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8d701579b0 tx=0x7f8d6800ff70 comp rx=0 tx=0).stop 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 shutdown_connections 2024-05-09T08:57:06.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d5406be30 0x7f8d5406e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d701508a0 0x7f8d701eff20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d70146f70 0x7f8d70150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 --2- 172.21.15.49:0/2462274312 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c003b40 0x7f8d7014f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 >> 172.21.15.49:0/2462274312 conn(0x7f8d7006b200 msgr2=0x7f8d70148140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 shutdown_connections 2024-05-09T08:57:06.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:06.371+0000 7f8d78316640 1 -- 172.21.15.49:0/2462274312 wait complete. 2024-05-09T08:57:06.417 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:06.418+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:06.441 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.441 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.450 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:06.518 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:06.519+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:06.559 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.560 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.568 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:06.619 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:06.620+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:06.683 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.683 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.692 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:06.720 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:06.721+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:06.821 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:06.822+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:06.836 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.836 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:06.922 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:06.923+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:06.948 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:06.948 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:06.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:07.023 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:07.024+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:07.059 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:07.059 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:07.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:07.125 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:07.126+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:07.171 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:07.171 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:07.180 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:07.264 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:07.265+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:07.323 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:07.323 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:07.327 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:07.328+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:07.331 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:07.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:07.428 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:07.429+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:07.476 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:07.476 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:07.485 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:07.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.523+0000 7f4bce2ee640 1 Processor -- start 2024-05-09T08:57:07.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.523+0000 7f4bce2ee640 1 -- start start 2024-05-09T08:57:07.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.523+0000 7f4bce2ee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 0x7f4bc8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.523+0000 7f4bce2ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc80c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bce2ee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc8159b10 con 0x7f4bc8152e30 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bce2ee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc8159270 con 0x7f4bc80c7130 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bce2ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc8159550 con 0x7f4bc8072670 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc7fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4bc8152e30 0x7f4bc80c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:43960/0 (socket says 172.21.15.49:43960) 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc7fff640 1 -- 172.21.15.49:0/1421498428 learned_addr learned my addr 172.21.15.49:0/1421498428 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/1421498428 >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 0x7f4bc8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/1421498428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc80c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587355660 0 0) 0x7f4bc8159b10 con 0x7f4bc8152e30 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bb0003520 con 0x7f4bc8152e30 2024-05-09T08:57:07.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bcc88b640 1 -- 172.21.15.49:0/1421498428 >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 msgr2=0x7f4bc8158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361277308 0 0) 0x7f4bb0003520 con 0x7f4bc8152e30 2024-05-09T08:57:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/1421498428 >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 0x7f4bc8158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bcc88b640 1 -- 172.21.15.49:0/1421498428 >> v1:172.21.15.49:6789/0 conn(0x7f4bc8152e30 legacy=0x7f4bc80c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bcc88b640 1 -- 172.21.15.49:0/1421498428 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc8159830 con 0x7f4bc8072670 2024-05-09T08:57:07.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.524+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/1421498428 >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 0x7f4bc8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:07.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.525+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/1421498428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc80c3790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009d90 tx=0x7f4bc002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e225dc96bf82d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.525+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8001670 con 0x7f4bc8072670 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.525+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc0007540 con 0x7f4bc8072670 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.525+0000 7f4bc6ffd640 1 -- 172.21.15.49:0/1421498428 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00078f0 con 0x7f4bc8072670 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 -- 172.21.15.49:0/1421498428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 msgr2=0x7f4bc80c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/1421498428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc80c3790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009d90 tx=0x7f4bc002a560 comp rx=0 tx=0).stop 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 -- 172.21.15.49:0/1421498428 shutdown_connections 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/1421498428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc80c3790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/1421498428 >> v2:172.21.15.183:3300/0 conn(0x7f4bc80c7130 0x7f4bc8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 -- 172.21.15.49:0/1421498428 >> 172.21.15.49:0/1421498428 conn(0x7f4bc80cc290 msgr2=0x7f4bc80cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 -- 172.21.15.49:0/1421498428 shutdown_connections 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.526+0000 7f4bce2ee640 1 -- 172.21.15.49:0/1421498428 wait complete. 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.527+0000 7f4bce2ee640 1 Processor -- start 2024-05-09T08:57:07.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.527+0000 7f4bce2ee640 1 -- start start 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc81df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc8152e30 0x7f4bc80c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc80c6620 con 0x7f4bc80c7130 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc80c59f0 con 0x7f4bc8152e30 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bce2ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc806b4b0 con 0x7f4bc8072670 2024-05-09T08:57:07.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49936/0 (socket says 172.21.15.49:49936) 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 -- 172.21.15.49:0/2174456076 learned_addr learned my addr 172.21.15.49:0/2174456076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 msgr2=0x7f4bc81df3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bc8072670 0x7f4bc81df3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc8152e30 msgr2=0x7f4bc80c3bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bc8152e30 0x7f4bc80c3bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc814f570 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e960 tx=0x7f4bbc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.528+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc017070 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.529+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bbc00ca00 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.529+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc814e8c0 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.529+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00cdd0 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.529+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc81f3d40 con 0x7f4bc80c7130 2024-05-09T08:57:07.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.529+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4bc804f660 con 0x7f4bc80c7130 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.530+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bbc01a070 con 0x7f4bc80c7130 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.530+0000 7f4bc4ff9640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 0x7f4bb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5857+0+0 (secure 0 0 0) 0x7f4bbc08ae40 con 0x7f4bc80c7130 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v36) ==== 54+0+16 (secure 0 0 0) 0x7f4bbc08d4c0 con 0x7f4bc80c7130 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4bc8151a00 0x7f4bc8148640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc80c3470 con 0x7f4bc8151a00 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4bc8151a00 0x7f4bc8148640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.531+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4bc8151a00 0x7f4bc8148640 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.532+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 0x7f4bb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.532+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc80c3470 con 0x7f4bc8151a00 2024-05-09T08:57:07.534 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:07.530+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:07.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.535+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 0x7f4bb406e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80011e0 tx=0x7f4bb8001110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.547+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc8151140 con 0x7f4bc8151a00 2024-05-09T08:57:07.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.549+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc8151140 con 0x7f4bc8151a00 2024-05-09T08:57:07.549 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.549+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4bc81f0520 0x7f4bc81f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.549+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4bc814f2f0 con 0x7f4bc81f0520 2024-05-09T08:57:07.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.549+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4bc81f0520 0x7f4bc81f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.549+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4bc81f0520 0x7f4bc81f0900 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.550+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc814f2f0 con 0x7f4bc81f0520 2024-05-09T08:57:07.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.565+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc8151e00 con 0x7f4bc81f0520 2024-05-09T08:57:07.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.566+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc8151e00 con 0x7f4bc81f0520 2024-05-09T08:57:07.566 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:07.567 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:07.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.567+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4bc8158230 0x7f4bc8061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.567+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4bc8152010 con 0x7f4bc8158230 2024-05-09T08:57:07.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.567+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4bc8158230 0x7f4bc8061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.567+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4bc8158230 0x7f4bc8061c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.568+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc8152010 con 0x7f4bc8158230 2024-05-09T08:57:07.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.582+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc80621b0 con 0x7f4bc8158230 2024-05-09T08:57:07.582 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.583+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc80621b0 con 0x7f4bc8158230 2024-05-09T08:57:07.583 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:07.583 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:07.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.583+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4bc80ca270 0x7f4bc80ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.583+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4bc80cabd0 con 0x7f4bc80ca270 2024-05-09T08:57:07.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.584+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4bc80ca270 0x7f4bc80ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.584+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4bc80ca270 0x7f4bc80ca690 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.584+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc80cabd0 con 0x7f4bc80ca270 2024-05-09T08:57:07.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.599+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc80cb030 con 0x7f4bc80ca270 2024-05-09T08:57:07.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.600+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc80cb030 con 0x7f4bc80ca270 2024-05-09T08:57:07.600 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:07.600 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:07.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.600+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4bc802c550 0x7f4bc802c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.600+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4bc80cb240 con 0x7f4bc802c550 2024-05-09T08:57:07.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.601+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4bc802c550 0x7f4bc802c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.601+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4bc802c550 0x7f4bc802c9b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.603+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc80cb240 con 0x7f4bc802c550 2024-05-09T08:57:07.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.617+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc80cb240 con 0x7f4bc802c550 2024-05-09T08:57:07.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.620+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc80cb240 con 0x7f4bc802c550 2024-05-09T08:57:07.619 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:07.620 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:07.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.620+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4bc8204c60 0x7f4bc8207050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.620+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4bc802d400 con 0x7f4bc8204c60 2024-05-09T08:57:07.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.620+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4bc8204c60 0x7f4bc8207050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.620+0000 7f4bc77fe640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4bc8204c60 0x7f4bc8207050 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.621+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc802d400 con 0x7f4bc8204c60 2024-05-09T08:57:07.630 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:07.631+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:07.632 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:07.632 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:07.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.635+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc80cb1e0 con 0x7f4bc8204c60 2024-05-09T08:57:07.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.637+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc80cb1e0 con 0x7f4bc8204c60 2024-05-09T08:57:07.636 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.637+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4bc8207aa0 0x7f4bc8209e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.637+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4bc820cc60 con 0x7f4bc8207aa0 2024-05-09T08:57:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.637+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4bc8207aa0 0x7f4bc8209e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.638+0000 7f4bcc88b640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4bc8207aa0 0x7f4bc8209e90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.639+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc820cc60 con 0x7f4bc8207aa0 2024-05-09T08:57:07.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:07.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.653+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc820a3d0 con 0x7f4bc8207aa0 2024-05-09T08:57:07.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.655+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc820a3d0 con 0x7f4bc8207aa0 2024-05-09T08:57:07.654 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:07.655 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:07.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.655+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4bc820c2b0 0x7f4bc820c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:07.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.655+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4bc8210000 con 0x7f4bc820c2b0 2024-05-09T08:57:07.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.656+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4bc820c2b0 0x7f4bc820c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:07.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.656+0000 7f4bc7fff640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4bc820c2b0 0x7f4bc820c6f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:07.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.657+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc8210000 con 0x7f4bc820c2b0 2024-05-09T08:57:07.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.671+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4bc820ee80 con 0x7f4bc820c2b0 2024-05-09T08:57:07.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.673+0000 7f4bc4ff9640 1 -- 172.21.15.49:0/2174456076 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4bc820ee80 con 0x7f4bc820c2b0 2024-05-09T08:57:07.673 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:07.674 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:07.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4bc8151a00 msgr2=0x7f4bc8148640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f4bc8151a00 0x7f4bc8148640 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4bc81f0520 msgr2=0x7f4bc81f0900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f4bc81f0520 0x7f4bc81f0900 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4bc8158230 msgr2=0x7f4bc8061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f4bc8158230 0x7f4bc8061c70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4bc80ca270 msgr2=0x7f4bc80ca690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f4bc80ca270 0x7f4bc80ca690 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4bc802c550 msgr2=0x7f4bc802c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f4bc802c550 0x7f4bc802c9b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4bc8204c60 msgr2=0x7f4bc8207050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f4bc8204c60 0x7f4bc8207050 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4bc8207aa0 msgr2=0x7f4bc8209e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f4bc8207aa0 0x7f4bc8209e90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bcc88b640 1 -- 172.21.15.49:0/2174456076 reap_dead start 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bcc88b640 1 -- 172.21.15.49:0/2174456076 reap_dead start 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4bc820c2b0 msgr2=0x7f4bc820c6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f4bc820c2b0 0x7f4bc820c6f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bcc88b640 1 -- 172.21.15.49:0/2174456076 reap_dead start 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 msgr2=0x7f4bb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 0x7f4bb406e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80011e0 tx=0x7f4bb8001110 comp rx=0 tx=0).stop 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 msgr2=0x7f4bc80c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e960 tx=0x7f4bbc00ee20 comp rx=0 tx=0).stop 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 shutdown_connections 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4bb406be30 0x7f4bb406e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 --2- 172.21.15.49:0/2174456076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bc80c7130 0x7f4bc80c3690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 >> 172.21.15.49:0/2174456076 conn(0x7f4bc80cc290 msgr2=0x7f4bc80c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 shutdown_connections 2024-05-09T08:57:07.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:07.674+0000 7f4bce2ee640 1 -- 172.21.15.49:0/2174456076 wait complete. 2024-05-09T08:57:07.731 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:07.732+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.105093369999999, 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:07.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:07.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.791 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.105090503, 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:07.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:07.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.797 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.105086858, 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:07.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.801 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.062277692999999, 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:07.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:07.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.807 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 56.057485704000001, 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:07.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:07.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.813 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.054979336000002, 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:07.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:07.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.817 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.053507946000003, 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:07.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.822 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 50.053432108000003, 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:07.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040610698000002, 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.827 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:07.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040589699999998, 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.833 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:07.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:07.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040585305999997, 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:07.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:07.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040580247999998, 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:07.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:07.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.850 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040576766999997, 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040572148999999, 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.856 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:07.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:07.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:07.861 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.04056868, 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:07.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:07.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.867 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.040565379, 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:07.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:07.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:07.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.04025231, 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:07.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.024176488000002, 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:07.878 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:07.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:07.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.024171809000002, 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:07.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.024165791000001, 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:07.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:07.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:07.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:07.895 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:07.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:07.897 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:07.898+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:07.998 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:07.999+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:08.041 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:08.042 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:08.050 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:08.099 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:08.100+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:08.193 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:08.193 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:08.193 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:08.193 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:08.193 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:08.199 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:08.200+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:08.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:08.301 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:08.302+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:08.345 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.682118857000006, 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:08.346 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:08.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:08.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:08.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:08.353 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289462592999996, 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:08.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289415000999995, 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.360 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:08.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:08.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289365133999993, 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.367 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:08.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289359333999997, 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:08.373 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:08.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:08.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289247533999998, 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:08.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289150371999995, 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:08.386 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:08.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.392 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289144613999994, 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:08.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:08.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.398 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289070973999998, 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.399 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:08.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.404 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289019162000002, 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.405 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:08.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.289015442999997, 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:08.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.803958985999998, 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:08.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:08.424 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.803942501999998, 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:08.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:08.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.803860545000006, 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:08.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:08.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.803812984999993, 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:08.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:08.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:08.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 44.600184540999997, 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:08.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:08.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:08.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.584105096000002, 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:08.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:08.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.58410078, 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:08.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:08.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.584091817000001, 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:08.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 38.584087756999999, 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:08.473 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:08.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:57:08.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:08.476 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:08.476+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:08.576 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:08.577+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:08.577 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:57:08.577 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:08.577 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-09T08:57:08.577 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-05-09T08:57:08.578 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-09T08:57:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.594+0000 7f5306e9e640 1 Processor -- start 2024-05-09T08:57:08.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.594+0000 7f5306e9e640 1 -- start start 2024-05-09T08:57:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5306e9e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5300154b90 0x7f53000cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5306e9e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5300154f70 0x7f53000cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5306e9e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5300072600 con 0x7f53000c6640 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5306e9e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f53000728e0 con 0x7f5300154f70 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5306e9e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f53000c5880 con 0x7f5300154b90 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5300154f70 0x7f53000cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5300154f70 0x7f53000cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49952/0 (socket says 172.21.15.49:49952) 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 -- 172.21.15.49:0/2197896478 learned_addr learned my addr 172.21.15.49:0/2197896478 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f5304c3a640 1 --2- 172.21.15.49:0/2197896478 >> v2:172.21.15.183:3300/0 conn(0x7f5300154b90 0x7f53000cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 -- 172.21.15.49:0/2197896478 >> v2:172.21.15.183:3300/0 conn(0x7f5300154b90 msgr2=0x7f53000cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 --2- 172.21.15.49:0/2197896478 >> v2:172.21.15.183:3300/0 conn(0x7f5300154b90 0x7f53000cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 -- 172.21.15.49:0/2197896478 >> v1:172.21.15.183:6789/0 conn(0x7f53000c6640 legacy=0x7f53000c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52fffff640 1 -- 172.21.15.49:0/2197896478 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000c5b60 con 0x7f5300154f70 2024-05-09T08:57:08.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.595+0000 7f52ff7fe640 1 -- 172.21.15.49:0/2197896478 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259150722 0 0) 0x7f5300072600 con 0x7f53000c6640 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.596+0000 7f52fffff640 1 --2- 172.21.15.49:0/2197896478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154f70 0x7f53000cb7c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f53000cdf50 tx=0x7f52ec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d5dbfbdd711ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.596+0000 7f52ff7fe640 1 -- 172.21.15.49:0/2197896478 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec037070 con 0x7f5300154f70 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.596+0000 7f52ff7fe640 1 -- 172.21.15.49:0/2197896478 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ec02ae50 con 0x7f5300154f70 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.596+0000 7f52ff7fe640 1 -- 172.21.15.49:0/2197896478 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec032860 con 0x7f5300154f70 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 -- 172.21.15.49:0/2197896478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154f70 msgr2=0x7f53000cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2197896478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154f70 0x7f53000cb7c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f53000cdf50 tx=0x7f52ec02a300 comp rx=0 tx=0).stop 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 -- 172.21.15.49:0/2197896478 shutdown_connections 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2197896478 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154f70 0x7f53000cb7c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2197896478 >> v2:172.21.15.183:3300/0 conn(0x7f5300154b90 0x7f53000cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 -- 172.21.15.49:0/2197896478 >> 172.21.15.49:0/2197896478 conn(0x7f5300074840 msgr2=0x7f5300074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:08.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 -- 172.21.15.49:0/2197896478 shutdown_connections 2024-05-09T08:57:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.597+0000 7f5306e9e640 1 -- 172.21.15.49:0/2197896478 wait complete. 2024-05-09T08:57:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.599+0000 7f5306e9e640 1 Processor -- start 2024-05-09T08:57:08.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.599+0000 7f5306e9e640 1 -- start start 2024-05-09T08:57:08.599 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:08.599 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:08.599 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:08.599 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 0x7f53001f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 0x7f53001ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5300154f70 0x7f53001ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f530015af40 con 0x7f5300154b90 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53000c6120 con 0x7f53000c6640 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f5306e9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5300071820 con 0x7f5300154f70 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f52fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 0x7f53001ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f52fffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 0x7f53001ee3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49958/0 (socket says 172.21.15.49:49958) 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f52fffff640 1 -- 172.21.15.49:0/2370166661 learned_addr learned my addr 172.21.15.49:0/2370166661 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:08.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 0x7f53001f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.600+0000 7f530543b640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5300154f70 0x7f53001ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 -- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5300154f70 msgr2=0x7f53001ee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5300154f70 0x7f53001ee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 -- 172.21.15.49:0/2370166661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 msgr2=0x7f53001ee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 0x7f53001ee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 -- 172.21.15.49:0/2370166661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000b5e50 con 0x7f53000c6640 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5304c3a640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 0x7f53001f3bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e880 tx=0x7f52f400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4017070 con 0x7f53000c6640 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f400c940 con 0x7f53000c6640 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400cd30 con 0x7f53000c6640 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.601+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001f46b0 con 0x7f53000c6640 2024-05-09T08:57:08.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.602+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53000cf430 con 0x7f53000c6640 2024-05-09T08:57:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.603+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f530004f660 con 0x7f53000c6640 2024-05-09T08:57:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.603+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52f401a070 con 0x7f53000c6640 2024-05-09T08:57:08.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.603+0000 7f52fd7fa640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 0x7f52e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.604+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5857+0+0 (secure 0 0 0) 0x7f52f408b290 con 0x7f53000c6640 2024-05-09T08:57:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.604+0000 7f52fffff640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 0x7f52e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.605+0000 7f52fffff640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 0x7f52e406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52ec035040 tx=0x7f52ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.608+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52f4059460 con 0x7f53000c6640 2024-05-09T08:57:08.610 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:08.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.690+0000 7f515a664640 1 Processor -- start 2024-05-09T08:57:08.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 -- start start 2024-05-09T08:57:08.690 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:08.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5154154b90 0x7f51540cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5154154f70 0x7f51540cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5154072600 con 0x7f51540c6640 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f51540728e0 con 0x7f5154154f70 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.691+0000 7f515a664640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f51540c5880 con 0x7f5154154b90 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5154154f70 0x7f51540cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5154154f70 0x7f51540cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49974/0 (socket says 172.21.15.49:49974) 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 -- 172.21.15.49:0/3076079839 learned_addr learned my addr 172.21.15.49:0/3076079839 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f5153fff640 1 --2- 172.21.15.49:0/3076079839 >> v2:172.21.15.183:3300/0 conn(0x7f5154154b90 0x7f51540cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 -- 172.21.15.49:0/3076079839 >> v2:172.21.15.183:3300/0 conn(0x7f5154154b90 msgr2=0x7f51540cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 --2- 172.21.15.49:0/3076079839 >> v2:172.21.15.183:3300/0 conn(0x7f5154154b90 0x7f51540cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 -- 172.21.15.49:0/3076079839 >> v1:172.21.15.49:6789/0 conn(0x7f51540c6640 legacy=0x7f51540c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.692+0000 7f51537fe640 1 -- 172.21.15.49:0/3076079839 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51540c5b60 con 0x7f5154154f70 2024-05-09T08:57:08.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.693+0000 7f51537fe640 1 --2- 172.21.15.49:0/3076079839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5154154f70 0x7f51540cb7c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51540cdf50 tx=0x7f514802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50551486c244eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.693+0000 7f5152ffd640 1 -- 172.21.15.49:0/3076079839 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148037070 con 0x7f5154154f70 2024-05-09T08:57:08.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.693+0000 7f5152ffd640 1 -- 172.21.15.49:0/3076079839 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514802ae50 con 0x7f5154154f70 2024-05-09T08:57:08.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.693+0000 7f5152ffd640 1 -- 172.21.15.49:0/3076079839 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5148032860 con 0x7f5154154f70 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 -- 172.21.15.49:0/3076079839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5154154f70 msgr2=0x7f51540cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 --2- 172.21.15.49:0/3076079839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5154154f70 0x7f51540cb7c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f51540cdf50 tx=0x7f514802a300 comp rx=0 tx=0).stop 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 -- 172.21.15.49:0/3076079839 shutdown_connections 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 --2- 172.21.15.49:0/3076079839 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5154154f70 0x7f51540cb7c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 --2- 172.21.15.49:0/3076079839 >> v2:172.21.15.183:3300/0 conn(0x7f5154154b90 0x7f51540cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 -- 172.21.15.49:0/3076079839 >> 172.21.15.49:0/3076079839 conn(0x7f5154074840 msgr2=0x7f5154074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 -- 172.21.15.49:0/3076079839 shutdown_connections 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.694+0000 7f515a664640 1 -- 172.21.15.49:0/3076079839 wait complete. 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.695+0000 7f515a664640 1 Processor -- start 2024-05-09T08:57:08.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.695+0000 7f515a664640 1 -- start start 2024-05-09T08:57:08.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 0x7f51541f09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 0x7f51541f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f515415af40 con 0x7f51540c6640 2024-05-09T08:57:08.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51540c6120 con 0x7f5154154b90 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f515a664640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5154071820 con 0x7f5154154f70 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f5153fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f5153fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49990/0 (socket says 172.21.15.49:49990) 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f5153fff640 1 -- 172.21.15.49:0/3914475388 learned_addr learned my addr 172.21.15.49:0/3914475388 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f5158c01640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 0x7f51541f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.696+0000 7f51537fe640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 0x7f51541f09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 -- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 msgr2=0x7f51541f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 0x7f51541f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 -- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 msgr2=0x7f51541f09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 0x7f51541f09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 -- 172.21.15.49:0/3914475388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51541f3ca0 con 0x7f51540c6640 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5158c01640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 0x7f51541f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:08.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f51537fe640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 0x7f51541f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5153fff640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f514400e9b0 tx=0x7f514400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144017070 con 0x7f51540c6640 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514400c9a0 con 0x7f51540c6640 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541f3f80 con 0x7f51540c6640 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514400cd90 con 0x7f51540c6640 2024-05-09T08:57:08.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.697+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541f4540 con 0x7f51540c6640 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.699+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515404f660 con 0x7f51540c6640 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.699+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f514401a070 con 0x7f51540c6640 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.700+0000 7f5150ff9640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 0x7f514006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.700+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5857+0+0 (secure 0 0 0) 0x7f514408b1c0 con 0x7f51540c6640 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.700+0000 7f51537fe640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 0x7f514006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:08.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.701+0000 7f51537fe640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 0x7f514006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f514802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:08.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.705+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5144059510 con 0x7f51540c6640 2024-05-09T08:57:08.765 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:08.765 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:08.765 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:08.765 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:08.766 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:08.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:08.778 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:08.779+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:08.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.805+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f53000cbde0 con 0x7f53000c6640 2024-05-09T08:57:08.879 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:08.880+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:08.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:08.904+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7f51540cbde0 con 0x7f51540c6640 2024-05-09T08:57:08.915 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:08.915 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.236024397000001, 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:08.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:08.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.236018827999999, 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:08.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:08.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:08.927 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.235987139999999, 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:08.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:08.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:08.933 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.235982454000002, 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:08.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:08.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:08.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.939 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.235978625999998, 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:08.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:08.943 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.23509576, 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:08.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:08.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:08.949 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.189522875000002, 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:08.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.955 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.188850100000003, 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:08.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:08.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.186129432999998, 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:08.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:08.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.186121391999997, 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:08.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:08.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.971 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.186117072000002, 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:08.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.186112766999997, 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:08.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:08.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.186109518999999, 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:08.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:08.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:08.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.171037146000003, 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:08.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:08.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:08.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.171025899, 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:08.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:08.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:08.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.171021478, 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:08.999 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:09.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:09.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.004 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.154917513000001, 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:09.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:09.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:09.010 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.154906181999998, 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:09.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:09.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:09.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.154898232000001, 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:09.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.021 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.154892353999998, 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:09.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:09.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.026 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.026 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:09.029 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:09.030+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:09.129 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:09.130+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:09.140 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:09.140 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:09.140 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.149 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:09.230 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:09.231+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:09.263 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:09.363 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:09.364+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.376+0000 7f5150ff9640 1 -- 172.21.15.49:0/3914475388 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v37) ==== 133+0+0 (secure 0 0 0) 0x7f514405d810 con 0x7f51540c6640 2024-05-09T08:57:09.375 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.378+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 msgr2=0x7f514006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.378+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 0x7f514006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5148034000 tx=0x7f514802e000 comp rx=0 tx=0).stop 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.378+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 msgr2=0x7f5154158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.378+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f514400e9b0 tx=0x7f514400ee70 comp rx=0 tx=0).stop 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 shutdown_connections 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f514006be30 0x7f514006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5154154f70 0x7f51541f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5154154b90 0x7f51541f09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 --2- 172.21.15.49:0/3914475388 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f51540c6640 0x7f5154158db0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 >> 172.21.15.49:0/3914475388 conn(0x7f5154074840 msgr2=0x7f51540c6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 shutdown_connections 2024-05-09T08:57:09.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f52fd7fa640 1 -- 172.21.15.49:0/2370166661 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v37) ==== 97+0+0 (secure 0 0 0) 0x7f52f405d760 con 0x7f53000c6640 2024-05-09T08:57:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.379+0000 7f515a664640 1 -- 172.21.15.49:0/3914475388 wait complete. 2024-05-09T08:57:09.380 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:57:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 msgr2=0x7f52e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 0x7f52e406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52ec035040 tx=0x7f52ec02e000 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 msgr2=0x7f53001f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 0x7f53001f3bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52f400e880 tx=0x7f52f400ed40 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 shutdown_connections 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f52e406be30 0x7f52e406e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5300154f70 0x7f53001ee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5300154b90 0x7f53001ee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 --2- 172.21.15.49:0/2370166661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53000c6640 0x7f53001f3bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 >> 172.21.15.49:0/2370166661 conn(0x7f5300074840 msgr2=0x7f53000cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 shutdown_connections 2024-05-09T08:57:09.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.381+0000 7f5306e9e640 1 -- 172.21.15.49:0/2370166661 wait complete. 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.755693684000001, 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.755669331999997, 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.755663577, 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.755648346000001, 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.712310361, 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.712302399999999, 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.472 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.705094371000001, 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:09.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.705083625999997, 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:09.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:09.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.484 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691109052999998, 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:09.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:09.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:09.490 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691098943999997, 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:09.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:09.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691096305999999, 2024-05-09T08:57:09.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:09.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.502 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691080704000001, 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:09.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.508 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691078054999998, 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:09.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:09.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:09.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691063929000002, 2024-05-09T08:57:09.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:09.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:09.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:09.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691061650000002, 2024-05-09T08:57:09.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:09.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:09.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:09.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:09.526 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691053744999998, 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:09.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:09.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.691052224000003, 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:09.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.670976375000002, 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:09.535 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:09.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.670954915000003, 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:09.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:09.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.547 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.670947910000002, 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:09.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:09.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:09.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:09.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:09.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:09.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.556 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:09.556 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:09.556 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.557 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:09.439+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:09.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:57:09.559 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:09.559+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:09.658 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:09.659+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:09.686 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:09.710 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:09.710 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:09.710 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:09.710 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.720 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:57:09.759 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:09.760+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:09.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.790+0000 7fbf92c12640 1 Processor -- start 2024-05-09T08:57:09.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.790+0000 7fbf92c12640 1 -- start start 2024-05-09T08:57:09.790 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf92c12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf92c12640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf8c0c6d50 0x7fbf8c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf92c12640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8c072600 con 0x7fbf8c154f70 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf92c12640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8c0728e0 con 0x7fbf8c0c6d50 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf92c12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c0c5880 con 0x7fbf8c154b90 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf909ae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf911af640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf8c0c6d50 0x7fbf8c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf909ae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:54190/0 (socket says 172.21.15.49:54190) 2024-05-09T08:57:09.791 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.791+0000 7fbf909ae640 1 -- 172.21.15.49:0/1188030607 learned_addr learned my addr 172.21.15.49:0/1188030607 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.792+0000 7fbf909ae640 1 -- 172.21.15.49:0/1188030607 >> v2:172.21.15.183:3300/0 conn(0x7fbf8c0c6d50 msgr2=0x7fbf8c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.792+0000 7fbf909ae640 1 --2- 172.21.15.49:0/1188030607 >> v2:172.21.15.183:3300/0 conn(0x7fbf8c0c6d50 0x7fbf8c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.792+0000 7fbf909ae640 1 -- 172.21.15.49:0/1188030607 >> v1:172.21.15.183:6789/0 conn(0x7fbf8c154f70 legacy=0x7fbf8c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.792+0000 7fbf909ae640 1 -- 172.21.15.49:0/1188030607 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c0c5b60 con 0x7fbf8c154b90 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.792+0000 7fbf909ae640 1 --2- 172.21.15.49:0/1188030607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009870 tx=0x7fbf8002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9755e779344a7c66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf8b7fe640 1 -- 172.21.15.49:0/1188030607 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80037070 con 0x7fbf8c154b90 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf8b7fe640 1 -- 172.21.15.49:0/1188030607 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8002ae50 con 0x7fbf8c154b90 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf8b7fe640 1 -- 172.21.15.49:0/1188030607 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80032860 con 0x7fbf8c154b90 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 -- 172.21.15.49:0/1188030607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 msgr2=0x7fbf8c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 --2- 172.21.15.49:0/1188030607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009870 tx=0x7fbf8002a300 comp rx=0 tx=0).stop 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 -- 172.21.15.49:0/1188030607 shutdown_connections 2024-05-09T08:57:09.793 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 --2- 172.21.15.49:0/1188030607 >> v2:172.21.15.183:3300/0 conn(0x7fbf8c0c6d50 0x7fbf8c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 --2- 172.21.15.49:0/1188030607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c154b90 0x7fbf8c0cb280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 -- 172.21.15.49:0/1188030607 >> 172.21.15.49:0/1188030607 conn(0x7fbf8c074840 msgr2=0x7fbf8c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:09.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 -- 172.21.15.49:0/1188030607 shutdown_connections 2024-05-09T08:57:09.794 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.793+0000 7fbf92c12640 1 -- 172.21.15.49:0/1188030607 wait complete. 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.794+0000 7fbf92c12640 1 Processor -- start 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 -- start start 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c0c6d50 0x7fbf8c14b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf8c154f70 0x7fbf8c0b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c15af40 con 0x7fbf8c154b90 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c0c6120 con 0x7fbf8c154f70 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.795+0000 7fbf92c12640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8c071820 con 0x7fbf8c0c6d50 2024-05-09T08:57:09.796 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:49994/0 (socket says 172.21.15.49:49994) 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 -- 172.21.15.49:0/50987020 learned_addr learned my addr 172.21.15.49:0/50987020 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c0c6d50 0x7fbf8c14b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf8c154f70 0x7fbf8c0b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c0c6d50 msgr2=0x7fbf8c14b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf8c0c6d50 0x7fbf8c14b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf8c154f70 msgr2=0x7fbf8c0b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf8c154f70 0x7fbf8c0b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c14dad0 con 0x7fbf8c154b90 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800c3f0 tx=0x7fbf7800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780052d0 con 0x7fbf8c154b90 2024-05-09T08:57:09.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf78005490 con 0x7fbf8c154b90 2024-05-09T08:57:09.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78007260 con 0x7fbf8c154b90 2024-05-09T08:57:09.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c14c3e0 con 0x7fbf8c154b90 2024-05-09T08:57:09.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.796+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c1f17c0 con 0x7fbf8c154b90 2024-05-09T08:57:09.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.797+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbf8c04f660 con 0x7fbf8c154b90 2024-05-09T08:57:09.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.798+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbf78007400 con 0x7fbf8c154b90 2024-05-09T08:57:09.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.798+0000 7fbf897fa640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 0x7fbf7006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.798+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5857+0+0 (secure 0 0 0) 0x7fbf7808a3d0 con 0x7fbf8c154b90 2024-05-09T08:57:09.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 0x7fbf7006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7fbf7808ca50 con 0x7fbf8c154b90 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fbf8c15ab30 0x7fbf8c152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c0d13b0 con 0x7fbf8c15ab30 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fbf8c15ab30 0x7fbf8c152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 0x7fbf7006e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf80034000 tx=0x7fbf8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:09.800 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:09.799+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fbf8c15ab30 0x7fbf8c152aa0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:09.860 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:09.861+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:09.871 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:09.872 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:09.872 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:09.872 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:09.872 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:09.880 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:57:09.961 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:09.962+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352405578000003, 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005543915, 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680886+0000", 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680905+0000", 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9360000000000001e-05 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681012+0000", 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107077 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681023+0000", 2024-05-09T08:57:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1236000000000001e-05 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116939 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681272+0000", 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013183499999999999 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681397+0000", 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124456 2024-05-09T08:57:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685651+0000", 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042543770000000002 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686328+0000", 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067728299999999999 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686380+0000", 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2012000000000003e-05 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686425+0000", 2024-05-09T08:57:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4305999999999999e-05 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352376431000003, 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060063249999999999, 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680914+0000", 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7799999999999998e-06 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681059+0000", 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000144986 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681066+0000", 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4910000000000002e-06 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682105+0000", 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010389920000000001 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682185+0000", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0357000000000003e-05 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682323+0000", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828000000000001 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685656+0000", 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033325149999999999 2024-05-09T08:57:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686805+0000", 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001149529 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686877+0000", 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1303000000000005e-05 2024-05-09T08:57:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686916+0000", 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9461e-05 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352370655000001, 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006182725, 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.96e-07 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.61e-06 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00020153400000000001 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681129+0000", 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2620000000000004e-06 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682565+0000", 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014361650000000001 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682695+0000", 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012924400000000001 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682808+0000", 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113004 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685660+0000", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002852121 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686999+0000", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001338802 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687050+0000", 2024-05-09T08:57:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1625000000000001e-05 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687098+0000", 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8062000000000002e-05 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352366787000001, 2024-05-09T08:57:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063447829999999997, 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680920+0000", 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9200000000000002e-07 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680945+0000", 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4856000000000001e-05 2024-05-09T08:57:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681153+0000", 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000208385 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681161+0000", 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3189999999999995e-06 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683100+0000", 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019385590000000001 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683175+0000", 2024-05-09T08:57:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4972e-05 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683297+0000", 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012193699999999999 2024-05-09T08:57:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685664+0000", 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023667929999999998 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687171+0000", 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001507216 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687223+0000", 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2571999999999997e-05 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687264+0000", 2024-05-09T08:57:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0881999999999999e-05 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352339622000002, 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064597120000000003, 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:10.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1600000000000001e-07 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680950+0000", 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:10.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226572 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681193+0000", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6884e-05 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683542+0000", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023484650000000001 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683657+0000", 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115277 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683766+0000", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010891399999999999 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685668+0000", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001901743 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687313+0000", 2024-05-09T08:57:10.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0016458709999999999 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687362+0000", 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8625999999999998e-05 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687406+0000", 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4276000000000001e-05 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352335160000003, 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066291869999999999, 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4799999999999999e-07 2024-05-09T08:57:10.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680954+0000", 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6730000000000001e-06 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681306+0000", 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000351491 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681313+0000", 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9619999999999996e-06 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684111+0000", 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027984569999999999 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684202+0000", 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1195999999999999e-05 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684353+0000", 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015071299999999999 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686447+0000", 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020943440000000002 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687488+0000", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001040336 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687534+0000", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6774999999999999e-05 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687580+0000", 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5892e-05 2024-05-09T08:57:10.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352331065000001, 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006791755, 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.082 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680956+0000", 2024-05-09T08:57:10.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.34e-07 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680958+0000", 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-06 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681332+0000", 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00037391099999999999 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681338+0000", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2639999999999999e-06 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684606+0000", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032684599999999999 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684723+0000", 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011661499999999999 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684853+0000", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013001199999999999 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686475+0000", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001621883 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687653+0000", 2024-05-09T08:57:10.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001178438 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687705+0000", 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1805000000000003e-05 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687747+0000", 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2143e-05 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.352326711000003, 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069219210000000002, 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:10.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6399999999999998e-07 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680968+0000", 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3289999999999994e-06 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681351+0000", 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00038298100000000001 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681356+0000", 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.101e-06 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685141+0000", 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003785086 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685240+0000", 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8267e-05 2024-05-09T08:57:10.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685406+0000", 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000166644 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686653+0000", 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012463190000000001 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687796+0000", 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011430450000000001 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687849+0000", 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2923999999999997e-05 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687881+0000", 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2861000000000003e-05 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.35230224, 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048537759999999998, 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680987+0000", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4370000000000001e-06 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681414+0000", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00042719999999999998 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681421+0000", 2024-05-09T08:57:10.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.601e-06 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682060+0000", 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00063917100000000001 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682139+0000", 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8872e-05 2024-05-09T08:57:10.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682269+0000", 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013019400000000001 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684249+0000", 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001979794 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685731+0000", 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014824930000000001 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685788+0000", 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6986e-05 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685838+0000", 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9755000000000001e-05 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.099 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:20 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 58.309329409999997, 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012308524, 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723958+0000", 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.296e-06 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.395e-06 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723990+0000", 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6914999999999999e-05 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723999+0000", 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5129999999999996e-06 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724055+0000", 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6159000000000002e-05 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724166+0000", 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110532 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.736265+0000", 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012099713999999999 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:15 2.3 e20/17 2:c27f454c:::mds1_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 58.307303982999997, 2024-05-09T08:57:10.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061558979999999999, 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725983+0000", 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8000000000000004e-07 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725991+0000", 2024-05-09T08:57:10.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0860000000000002e-06 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726006+0000", 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4938e-05 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2429999999999999e-06 2024-05-09T08:57:10.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727846+0000", 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018329609999999999 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727871+0000", 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4406999999999999e-05 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732095+0000", 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042245640000000001 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732138+0000", 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818999999999998e-05 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.109 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303223246000002, 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057959939999999996, 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1199999999999998e-07 2024-05-09T08:57:10.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730066+0000", 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.694e-06 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730250+0000", 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018380499999999999 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730260+0000", 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7100000000000002e-06 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731230+0000", 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097010499999999997 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731303+0000", 2024-05-09T08:57:10.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3579999999999997e-05 2024-05-09T08:57:10.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:10.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731420+0000", 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116568 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732988+0000", 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015678440000000001 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735786+0000", 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027978769999999998 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:10.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1066000000000003e-05 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735859+0000", 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2332999999999997e-05 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.117 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303218848, 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048474440000000002, 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730068+0000", 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730070+0000", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.553e-06 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730295+0000", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00022485300000000001 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730301+0000", 2024-05-09T08:57:10.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9580000000000004e-06 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730352+0000", 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1400999999999998e-05 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730483+0000", 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013079300000000001 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010618799999999999 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733630+0000", 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030401299999999998 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734823+0000", 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011937950000000001 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734873+0000", 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9707999999999999e-05 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734915+0000", 2024-05-09T08:57:10.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1746999999999999e-05 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303214826000001, 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050848680000000002, 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730072+0000", 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6100000000000002e-07 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730075+0000", 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8490000000000002e-06 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730966+0000", 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089175099999999998 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730979+0000", 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2847000000000001e-05 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731802+0000", 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082315399999999999 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731879+0000", 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7046999999999996e-05 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732012+0000", 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000133005 2024-05-09T08:57:10.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733927+0000", 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001914612 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735049+0000", 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011219789999999999 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735113+0000", 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4362999999999993e-05 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735156+0000", 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2899999999999999e-05 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.130 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303207385, 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.004901702, 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730081+0000", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8220000000000001e-06 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731051+0000", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096957499999999997 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731057+0000", 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9560000000000002e-06 2024-05-09T08:57:10.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731629+0000", 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057226399999999995 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731714+0000", 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5661999999999994e-05 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732235+0000", 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052096200000000003 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733613+0000", 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013775339999999999 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734929+0000", 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001316198 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734954+0000", 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4477999999999998e-05 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734981+0000", 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6933000000000001e-05 2024-05-09T08:57:10.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303204149000003, 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053901399999999999, 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8700000000000002e-07 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730085+0000", 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7609999999999999e-06 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731070+0000", 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098512399999999998 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731083+0000", 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3033999999999999e-05 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732979+0000", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001895276 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733089+0000", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110542 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733211+0000", 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121471 2024-05-09T08:57:10.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734283+0000", 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001071904 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735386+0000", 2024-05-09T08:57:10.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011033620000000001 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735437+0000", 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0735999999999997e-05 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735472+0000", 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5642999999999997e-05 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.145 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303200040999997, 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054919699999999997, 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:10.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730087+0000", 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7599999999999997e-07 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730089+0000", 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731097+0000", 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001008412 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731102+0000", 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5589999999999999e-06 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732441+0000", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00133946 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732543+0000", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010173199999999999 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732613+0000", 2024-05-09T08:57:10.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9788000000000003e-05 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734007+0000", 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013943499999999999 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735476+0000", 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014689709999999999 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735536+0000", 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0408999999999997e-05 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735578+0000", 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1897000000000002e-05 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303196448999998, 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059167780000000001, 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:10.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5699999999999999e-07 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730096+0000", 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0599999999999996e-06 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731114+0000", 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001017525 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731118+0000", 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7639999999999996e-06 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731633+0000", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00051442600000000001 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731741+0000", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010863599999999999 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731906+0000", 2024-05-09T08:57:10.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016487199999999999 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733654+0000", 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001747738 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735909+0000", 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022545 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735950+0000", 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1201000000000001e-05 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.736007+0000", 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6799000000000003e-05 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303182587000002, 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055945150000000004, 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:10.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000002e-07 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730113+0000", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1020000000000003e-06 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731131+0000", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010174279999999999 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731135+0000", 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4449999999999998e-06 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733092+0000", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019572190000000001 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733229+0000", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013633499999999999 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733345+0000", 2024-05-09T08:57:10.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116647 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734293+0000", 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094760700000000005 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735618+0000", 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013248260000000001 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735667+0000", 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9468000000000001e-05 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735698+0000", 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1053000000000002e-05 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.163 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 52.303171763000002, 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057439600000000002, 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:10.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730115+0000", 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730117+0000", 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7940000000000001e-06 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731147+0000", 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010300610000000001 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731151+0000", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7269999999999998e-06 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733642+0000", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024900780000000002 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733758+0000", 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116214 2024-05-09T08:57:10.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735858+0000", 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021007040000000001 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:10.168 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:10.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.045+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c0d13b0 con 0x7fbf8c15ab30 2024-05-09T08:57:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.061+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c0cbde0 con 0x7fbf8c15ab30 2024-05-09T08:57:10.171 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.063+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c0cbde0 con 0x7fbf8c15ab30 2024-05-09T08:57:10.171 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:10.172 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.063+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbf8c14c5b0 0x7fbf8c1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.063+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c0cd0b0 con 0x7fbf8c14c5b0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.063+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbf8c14c5b0 0x7fbf8c1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.063+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbf8c14c5b0 0x7fbf8c1f3e90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.064+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c0cd0b0 con 0x7fbf8c14c5b0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.079+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c0b6230 con 0x7fbf8c14c5b0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.081+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c0b6230 con 0x7fbf8c14c5b0 2024-05-09T08:57:10.173 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.081+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fbf8c1f48e0 0x7fbf8c061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.081+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c1f4d00 con 0x7fbf8c1f48e0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.081+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fbf8c1f48e0 0x7fbf8c061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.081+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fbf8c1f48e0 0x7fbf8c061c70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.082+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c1f4d00 con 0x7fbf8c1f48e0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.097+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c0621b0 con 0x7fbf8c1f48e0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.099+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c0621b0 con 0x7fbf8c1f48e0 2024-05-09T08:57:10.175 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.099+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fbf8c0b4660 0x7fbf8c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.099+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c0b4fe0 con 0x7fbf8c0b4660 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.099+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fbf8c0b4660 0x7fbf8c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.099+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fbf8c0b4660 0x7fbf8c0b4aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.100+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c0b4fe0 con 0x7fbf8c0b4660 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.114+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c0b4fe0 con 0x7fbf8c0b4660 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.116+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c0b4fe0 con 0x7fbf8c0b4660 2024-05-09T08:57:10.177 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.116+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fbf8c02c550 0x7fbf8c02c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.116+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c0b4fe0 con 0x7fbf8c02c550 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.116+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fbf8c02c550 0x7fbf8c02c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.117+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fbf8c02c550 0x7fbf8c02c9d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.117+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c0b4fe0 con 0x7fbf8c02c550 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.132+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c0b4fe0 con 0x7fbf8c02c550 2024-05-09T08:57:10.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.134+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c0b4fe0 con 0x7fbf8c02c550 2024-05-09T08:57:10.180 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:10.171+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:10.180 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.134+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fbf8c205260 0x7fbf8c207650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.134+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c073ba0 con 0x7fbf8c205260 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.134+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fbf8c205260 0x7fbf8c207650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.135+0000 7fbf8bfff640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fbf8c205260 0x7fbf8c207650 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.135+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c073ba0 con 0x7fbf8c205260 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.149+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c073ba0 con 0x7fbf8c205260 2024-05-09T08:57:10.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.151+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c073ba0 con 0x7fbf8c205260 2024-05-09T08:57:10.182 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.152+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fbf8c2080a0 0x7fbf8c20a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.152+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c02d420 con 0x7fbf8c2080a0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.152+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fbf8c2080a0 0x7fbf8c20a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.152+0000 7fbf911af640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fbf8c2080a0 0x7fbf8c20a490 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.153+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c02d420 con 0x7fbf8c2080a0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.167+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c02d420 con 0x7fbf8c2080a0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.170+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c02d420 con 0x7fbf8c2080a0 2024-05-09T08:57:10.183 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.170+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fbf8c20aee0 0x7fbf8c20d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.170+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbf8c2101e0 con 0x7fbf8c20aee0 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.170+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fbf8c20aee0 0x7fbf8c20d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.170+0000 7fbf909ae640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fbf8c20aee0 0x7fbf8c20d2d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.171+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf8c2101e0 con 0x7fbf8c20aee0 2024-05-09T08:57:10.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.185+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbf8c20d810 con 0x7fbf8c20aee0 2024-05-09T08:57:10.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf897fa640 1 -- 172.21.15.49:0/50987020 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbf8c20d810 con 0x7fbf8c20aee0 2024-05-09T08:57:10.187 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fbf8c15ab30 msgr2=0x7fbf8c152aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fbf8c15ab30 0x7fbf8c152aa0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbf8c14c5b0 msgr2=0x7fbf8c1f3e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbf8c14c5b0 0x7fbf8c1f3e90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fbf8c1f48e0 msgr2=0x7fbf8c061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fbf8c1f48e0 0x7fbf8c061c70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fbf8c0b4660 msgr2=0x7fbf8c0b4aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fbf8c0b4660 0x7fbf8c0b4aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fbf8c02c550 msgr2=0x7fbf8c02c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fbf8c02c550 0x7fbf8c02c9d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fbf8c205260 msgr2=0x7fbf8c207650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fbf8c205260 0x7fbf8c207650 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fbf8c2080a0 msgr2=0x7fbf8c20a490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf911af640 1 -- 172.21.15.49:0/50987020 reap_dead start 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fbf8c2080a0 0x7fbf8c20a490 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf911af640 1 -- 172.21.15.49:0/50987020 reap_dead start 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fbf8c20aee0 msgr2=0x7fbf8c20d2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fbf8c20aee0 0x7fbf8c20d2d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf911af640 1 -- 172.21.15.49:0/50987020 reap_dead start 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 msgr2=0x7fbf7006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 0x7fbf7006e320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf80034000 tx=0x7fbf8002e000 comp rx=0 tx=0).stop 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 msgr2=0x7fbf8c14cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800c3f0 tx=0x7fbf7800c8b0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 shutdown_connections 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbf7006be60 0x7fbf7006e320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 --2- 172.21.15.49:0/50987020 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbf8c154b90 0x7fbf8c14cba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 >> 172.21.15.49:0/50987020 conn(0x7fbf8c074840 msgr2=0x7fbf8c14f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 shutdown_connections 2024-05-09T08:57:10.190 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:10.188+0000 7fbf92c12640 1 -- 172.21.15.49:0/50987020 wait complete. 2024-05-09T08:57:10.271 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:10.272+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:10.295 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.295 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:10.372 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:10.373+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:10.414 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.415 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:10.508 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:10.509+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:10.573 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.574 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.574 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:10.575+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:10.582 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:10.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:57:10.624 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:57:10.675 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:10.676+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:10.699 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.699 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.709 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:10.776 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:10.777+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:10.813 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.813 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:10.907 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:10.908+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:10.967 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:10.967 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:10.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:11.017 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:11.018+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:11.079 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:11.080+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:11.120 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.120 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:11.180 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:11.181+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:11.231 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.232 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:11.281 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:11.282+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:11.343 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.343 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.352 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:11.396 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:11.397+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:11.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:57:11.456 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.456 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.465 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:11.506 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:11.507+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:11.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.510+0000 7fd97fdd5640 1 Processor -- start 2024-05-09T08:57:11.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.510+0000 7fd97fdd5640 1 -- start start 2024-05-09T08:57:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd978155180 0x7fd9780c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9780c8f90 0x7fd97806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780c7b90 con 0x7fd97806b740 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd97815f160 con 0x7fd978155180 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97fdd5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780c78b0 con 0x7fd9780c8f90 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97e372640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97e372640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50008/0 (socket says 172.21.15.49:50008) 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97e372640 1 -- 172.21.15.49:0/449644049 learned_addr learned my addr 172.21.15.49:0/449644049 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 --2- 172.21.15.49:0/449644049 >> v2:172.21.15.183:3300/0 conn(0x7fd978155180 0x7fd9780c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97d370640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9780c8f90 0x7fd97806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 -- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9780c8f90 msgr2=0x7fd97806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9780c8f90 0x7fd97806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 -- 172.21.15.49:0/449644049 >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 msgr2=0x7fd9780c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 --2- 172.21.15.49:0/449644049 >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97db71640 1 -- 172.21.15.49:0/449644049 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97815e3a0 con 0x7fd978155180 2024-05-09T08:57:11.512 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.511+0000 7fd97e372640 1 --2- 172.21.15.49:0/449644049 >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:11.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.512+0000 7fd97db71640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9780c8950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd97806cef0 tx=0x7fd97402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56d9bc7a86648a98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:11.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.512+0000 7fd97cb6f640 1 -- 172.21.15.49:0/449644049 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974037070 con 0x7fd978155180 2024-05-09T08:57:11.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.512+0000 7fd97cb6f640 1 -- 172.21.15.49:0/449644049 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97402add0 con 0x7fd978155180 2024-05-09T08:57:11.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.512+0000 7fd97cb6f640 1 -- 172.21.15.49:0/449644049 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974032660 con 0x7fd978155180 2024-05-09T08:57:11.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 -- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 msgr2=0x7fd9780c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:11.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9780c8950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd97806cef0 tx=0x7fd97402a080 comp rx=0 tx=0).stop 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 -- 172.21.15.49:0/449644049 shutdown_connections 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/449644049 >> v2:172.21.15.49:3300/0 conn(0x7fd97806b740 0x7fd9780c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9780c8f90 0x7fd97806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/449644049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9780c8950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 -- 172.21.15.49:0/449644049 >> 172.21.15.49:0/449644049 conn(0x7fd9780c9650 msgr2=0x7fd9780c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 -- 172.21.15.49:0/449644049 shutdown_connections 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.513+0000 7fd97fdd5640 1 -- 172.21.15.49:0/449644049 wait complete. 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.514+0000 7fd97fdd5640 1 Processor -- start 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.514+0000 7fd97fdd5640 1 -- start start 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97806b740 0x7fd9781480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9781f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780c7550 con 0x7fd9780c8f90 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd97815e960 con 0x7fd978155180 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97fdd5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780c32a0 con 0x7fd97806b740 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50018/0 (socket says 172.21.15.49:50018) 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 -- 172.21.15.49:0/13987988 learned_addr learned my addr 172.21.15.49:0/13987988 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97db71640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97806b740 0x7fd9781480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97e372640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9781f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 -- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97806b740 msgr2=0x7fd9781480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97806b740 0x7fd9781480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 -- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 msgr2=0x7fd9781f0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9781f0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 -- 172.21.15.49:0/13987988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9781f3e70 con 0x7fd9780c8f90 2024-05-09T08:57:11.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.515+0000 7fd97d370640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd96800e9b0 tx=0x7fd96800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.516+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968017070 con 0x7fd9780c8f90 2024-05-09T08:57:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.516+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd96800c9a0 con 0x7fd9780c8f90 2024-05-09T08:57:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.516+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800cd90 con 0x7fd9780c8f90 2024-05-09T08:57:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.516+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9781f4130 con 0x7fd9780c8f90 2024-05-09T08:57:11.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.516+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781f46f0 con 0x7fd9780c8f90 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.517+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97804f660 con 0x7fd9780c8f90 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.517+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd96801a070 con 0x7fd9780c8f90 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.518+0000 7fd96e7fc640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 0x7fd95c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.518+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5857+0+0 (secure 0 0 0) 0x7fd96808ae60 con 0x7fd9780c8f90 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.518+0000 7fd97db71640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 0x7fd95c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:11.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.518+0000 7fd97db71640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 0x7fd95c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd974034000 tx=0x7fd97402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:11.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.522+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd968059120 con 0x7fd9780c8f90 2024-05-09T08:57:11.585 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:11.586+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:11.608 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.608 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.616 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:11.685 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:11.686+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:11.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:11.714+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fd9780c7d70 con 0x7fd9780c8f90 2024-05-09T08:57:11.719 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.719 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:11.812 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:11.813+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:11.871 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:11.871 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:11.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:11.887 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:11.888+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:11.988 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:11.989+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:12.022 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:12.023 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:12.031 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:12.089 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:12.090+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:12.174 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:12.174 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:12.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:12.199 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:12.227 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:12.228+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:12.291 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:12.292+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:12.296 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:12.296 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:12.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:12.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.331+0000 7f680ec9e640 1 Processor -- start 2024-05-09T08:57:12.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.331+0000 7f680ec9e640 1 -- start start 2024-05-09T08:57:12.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.332+0000 7f680ec9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.332+0000 7f680ec9e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f68080c9500 0x7f68080c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.332+0000 7f680ec9e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6808070990 con 0x7f68080733c0 2024-05-09T08:57:12.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.332+0000 7f680ec9e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f68080ca5f0 con 0x7f68080c9500 2024-05-09T08:57:12.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.332+0000 7f680ec9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f680815ccd0 con 0x7f6808152f30 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f680ca3a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f68080c9500 0x7f68080c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f680d23b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f68080733c0 0x7f6808072120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:40510/0 (socket says 172.21.15.49:40510) 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f680ca3a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:54230/0 (socket says 172.21.15.49:54230) 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f68080c9500 0x7f68080c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:57428/0 (socket says 172.21.15.49:57428) 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f680d23b640 1 -- 172.21.15.49:0/3638355738 learned_addr learned my addr 172.21.15.49:0/3638355738 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 -- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 msgr2=0x7f68080c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 -- 172.21.15.49:0/3638355738 >> v1:172.21.15.183:6789/0 conn(0x7f68080733c0 legacy=0x7f6808072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f68077fe640 1 -- 172.21.15.49:0/3638355738 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373275470 0 0) 0x7f6808070990 con 0x7f68080733c0 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f6807fff640 1 -- 172.21.15.49:0/3638355738 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680815cfb0 con 0x7f68080c9500 2024-05-09T08:57:12.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.333+0000 7f680ca3a640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.334+0000 7f6807fff640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68080c7600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67f40097a0 tx=0x7f67f402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa209103f7a2c9dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.334+0000 7f68077fe640 1 -- 172.21.15.49:0/3638355738 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4037070 con 0x7f68080c9500 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.334+0000 7f68077fe640 1 -- 172.21.15.49:0/3638355738 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f4032440 con 0x7f68080c9500 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.334+0000 7f68077fe640 1 -- 172.21.15.49:0/3638355738 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4032810 con 0x7f68080c9500 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 -- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 msgr2=0x7f68080c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68080c7600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67f40097a0 tx=0x7f67f402a210 comp rx=0 tx=0).stop 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 -- 172.21.15.49:0/3638355738 shutdown_connections 2024-05-09T08:57:12.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68080c7600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 --2- 172.21.15.49:0/3638355738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68080c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 -- 172.21.15.49:0/3638355738 >> 172.21.15.49:0/3638355738 conn(0x7f68080ce390 msgr2=0x7f68080ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 -- 172.21.15.49:0/3638355738 shutdown_connections 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.335+0000 7f680ec9e640 1 -- 172.21.15.49:0/3638355738 wait complete. 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.336+0000 7f680ec9e640 1 Processor -- start 2024-05-09T08:57:12.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.336+0000 7f680ec9e640 1 -- start start 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68081539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68081591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68080caa80 con 0x7f68080733c0 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68080c5dd0 con 0x7f68080c9500 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ec9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f68080ce010 con 0x7f6808152f30 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:50034/0 (socket says 172.21.15.49:50034) 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 -- 172.21.15.49:0/2872384076 learned_addr learned my addr 172.21.15.49:0/2872384076 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:12.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68081591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68081539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 msgr2=0x7f68081591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6808152f30 0x7f68081591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 msgr2=0x7f68081539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68080c9500 0x7f68081539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.337+0000 7f680ca3a640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680814e5f0 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00ebe0 tx=0x7f67fc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc017070 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67fc00cc30 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680814e8d0 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc013470 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.338+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68081eedf0 con 0x7f68080733c0 2024-05-09T08:57:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.339+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f68081ee900 con 0x7f68080733c0 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.339+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67fc01a070 con 0x7f68080733c0 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f68057fa640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 0x7f67ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5857+0+0 (secure 0 0 0) 0x7f67fc08b1b0 con 0x7f68080733c0 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7f67fc08d830 con 0x7f68080733c0 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 0x7f67ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f68081ef030 0x7f6808148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f680804f660 con 0x7f68081ef030 2024-05-09T08:57:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.340+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f68081ef030 0x7f6808148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.341+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f68081ef030 0x7f6808148ce0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.341+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 0x7f67ec06e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f67f4034000 tx=0x7f67f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.341+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f680804f660 con 0x7f68081ef030 2024-05-09T08:57:12.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.357+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f680814cf90 con 0x7f68081ef030 2024-05-09T08:57:12.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.358+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f680814cf90 con 0x7f68081ef030 2024-05-09T08:57:12.358 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.359+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f68081ef5f0 0x7f68081f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.359+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f68080731a0 con 0x7f68081ef5f0 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.359+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f68081ef5f0 0x7f68081f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.359+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f68081ef5f0 0x7f68081f1e70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.360+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68080731a0 con 0x7f68081ef5f0 2024-05-09T08:57:12.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.375+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6808153f30 con 0x7f68081ef5f0 2024-05-09T08:57:12.376 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.377+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6808153f30 con 0x7f68081ef5f0 2024-05-09T08:57:12.376 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:12.377 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.377+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f68081f28c0 0x7f6808061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.377+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f68081f2cc0 con 0x7f68081f28c0 2024-05-09T08:57:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.377+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f68081f28c0 0x7f6808061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.377+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f68081f28c0 0x7f6808061c70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.378 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.378+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.2 v2:172.21.15.49:6808/136930210 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67f4032960 con 0x7f68081f28c0 2024-05-09T08:57:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.380+0000 7fd96e7fc640 1 -- 172.21.15.49:0/13987988 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v38) ==== 91+0+0 (secure 0 0 0) 0x7fd96805d420 con 0x7fd9780c8f90 2024-05-09T08:57:12.379 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.382+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 msgr2=0x7fd95c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.382+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 0x7fd95c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd974034000 tx=0x7fd97402e000 comp rx=0 tx=0).stop 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.382+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 msgr2=0x7fd97814ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.382+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd96800e9b0 tx=0x7fd96800ee70 comp rx=0 tx=0).stop 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 shutdown_connections 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd95c06be30 0x7fd95c06e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd978155180 0x7fd9781f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd9780c8f90 0x7fd97814ab90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 --2- 172.21.15.49:0/13987988 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd97806b740 0x7fd9781480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 >> 172.21.15.49:0/13987988 conn(0x7fd9780c9650 msgr2=0x7fd9780cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:12.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 shutdown_connections 2024-05-09T08:57:12.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.383+0000 7fd97fdd5640 1 -- 172.21.15.49:0/13987988 wait complete. 2024-05-09T08:57:12.392 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:12.393+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:12.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.395+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6808062480 con 0x7f68081f28c0 2024-05-09T08:57:12.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.397+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.2 v2:172.21.15.49:6808/136930210 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6808062480 con 0x7f68081f28c0 2024-05-09T08:57:12.396 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.397+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f68080cca10 0x7f68080cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.397+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6808069050 con 0x7f68080cca10 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.397+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f68080cca10 0x7f68080cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.397+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f68080cca10 0x7f68080cce90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.398+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6808069050 con 0x7f68080cca10 2024-05-09T08:57:12.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.413+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6808069050 con 0x7f68080cca10 2024-05-09T08:57:12.413 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.415+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6808069050 con 0x7f68080cca10 2024-05-09T08:57:12.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:12.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.415+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f680802c550 0x7f680802c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.415+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f68080701b0 con 0x7f680802c550 2024-05-09T08:57:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.415+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f680802c550 0x7f680802c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.416+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f680802c550 0x7f680802c9b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.415 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.416+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68080701b0 con 0x7f680802c550 2024-05-09T08:57:12.422 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:12.422 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:12.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.431+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f68080701b0 con 0x7f680802c550 2024-05-09T08:57:12.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:12.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.433+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f68080701b0 con 0x7f680802c550 2024-05-09T08:57:12.432 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:12.433 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:12.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.433+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f68082031f0 0x7f68082055e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.433+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f680806fed0 con 0x7f68082031f0 2024-05-09T08:57:12.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.434+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f68082031f0 0x7f68082055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.434+0000 7f6807fff640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f68082031f0 0x7f68082055e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.435+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f680806fed0 con 0x7f68082031f0 2024-05-09T08:57:12.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.451+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f680806fed0 con 0x7f68082031f0 2024-05-09T08:57:12.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.453+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f680806fed0 con 0x7f68082031f0 2024-05-09T08:57:12.452 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.453+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6808206030 0x7f6808208420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.453+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f680802d480 con 0x7f6808206030 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.453+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6808206030 0x7f6808208420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.454+0000 7f680d23b640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6808206030 0x7f6808208420 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.454+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f680802d480 con 0x7f6808206030 2024-05-09T08:57:12.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.469+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f68080cd450 con 0x7f6808206030 2024-05-09T08:57:12.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.471+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f68080cd450 con 0x7f6808206030 2024-05-09T08:57:12.470 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.471+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6808208e70 0x7f680820b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.471+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6808062c30 con 0x7f6808208e70 2024-05-09T08:57:12.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.472+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6808208e70 0x7f680820b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.472+0000 7f680ca3a640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6808208e70 0x7f680820b260 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:12.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.473+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67fc061f20 con 0x7f6808208e70 2024-05-09T08:57:12.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.488+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f680820b820 con 0x7f6808208e70 2024-05-09T08:57:12.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.489+0000 7f68057fa640 1 -- 172.21.15.49:0/2872384076 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f680820b820 con 0x7f6808208e70 2024-05-09T08:57:12.489 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f68081ef030 msgr2=0x7f6808148ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f68081ef030 0x7f6808148ce0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f68081ef5f0 msgr2=0x7f68081f1e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f68081ef5f0 0x7f68081f1e70 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f68081f28c0 msgr2=0x7f6808061c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7f68081f28c0 0x7f6808061c70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f68080cca10 msgr2=0x7f68080cce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f68080cca10 0x7f68080cce90 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f680802c550 msgr2=0x7f680802c9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f680802c550 0x7f680802c9b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f68082031f0 msgr2=0x7f68082055e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f68082031f0 0x7f68082055e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6808206030 msgr2=0x7f6808208420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f6808206030 0x7f6808208420 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680d23b640 1 -- 172.21.15.49:0/2872384076 reap_dead start 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680d23b640 1 -- 172.21.15.49:0/2872384076 reap_dead start 2024-05-09T08:57:12.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6808208e70 msgr2=0x7f680820b260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6808208e70 0x7f680820b260 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680d23b640 1 -- 172.21.15.49:0/2872384076 reap_dead start 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 msgr2=0x7f67ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 0x7f67ec06e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f67f4034000 tx=0x7f67f402e000 comp rx=0 tx=0).stop 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 msgr2=0x7f68081560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00ebe0 tx=0x7f67fc00c520 comp rx=0 tx=0).stop 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 shutdown_connections 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f67ec06be30 0x7f67ec06e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 --2- 172.21.15.49:0/2872384076 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68080733c0 0x7f68081560a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 >> 172.21.15.49:0/2872384076 conn(0x7f68080ce390 msgr2=0x7f6808150df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 shutdown_connections 2024-05-09T08:57:12.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:12.490+0000 7f680ec9e640 1 -- 172.21.15.49:0/2872384076 wait complete. 2024-05-09T08:57:12.493 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:12.494+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:12.552 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.872426809999993, 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.872421240999998, 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.872389553000005, 2024-05-09T08:57:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.872384866999994, 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.872381039000004, 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.871498173000006, 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 60.825925288000001, 2024-05-09T08:57:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 60.825252513000002, 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.822531845999997, 2024-05-09T08:57:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.822523805000003, 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.822519485000001, 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.822515180000003, 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.822511931999998, 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.807439559000002, 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.807428311999999, 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 48.807423890999999, 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.791319926, 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.791308594999997, 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.791300645, 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.791294766999997, 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:12.681 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:12.682 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:12.683 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:12.683+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:12.783 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:12.784+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:12.834 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:12.834 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:12.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:12.884 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:12.885+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:12.984 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:12.986+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:12.996 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:12.997 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:13.006 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:13.090 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:13.091+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.479415613, 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:13.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:13.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:13.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.479391261000004, 2024-05-09T08:57:13.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:13.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:13.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.479385506, 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:13.174 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:13.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:13.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:13.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:13.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:13.181 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.479370274999994, 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.182 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:13.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:13.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.188 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.43603229, 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:13.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:13.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:13.194 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.436024328999999, 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:13.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:13.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.201 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.428816300000001, 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:13.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.208 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.428805554999997, 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:13.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:13.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.214 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414830981999998, 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:13.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.220 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414820872999996, 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:13.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:13.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.226 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414818234999998, 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:13.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:13.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:13.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:13.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.233 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414802633000001, 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:13.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414799983999998, 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:13.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:13.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.247 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414785858000002, 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:13.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414783579000002, 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:13.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:13.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.261 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414775673999998, 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:13.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.269 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 49.414774153000003, 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:13.271 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:13.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.275 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.394698304000002, 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:13.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.394676844000003, 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:13.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:13.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:13.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:13.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.394669839000002, 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:13.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.295 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.296 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:13.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:57:13.298 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:13.298+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:13.398 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:13.399+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:13.449 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:13.449 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:13.449 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:13.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-09T08:57:13.502 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:13.503+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:13.581 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:13.581 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:13.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-09T08:57:13.632 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:13.633+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:13.701 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:57:13.702+0000 7f18cb9b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 35658) UID: 0 2024-05-09T08:57:13.751 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:13.751 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:13.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070559744999997, 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005543915, 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.752 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680881+0000", 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680886+0000", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680905+0000", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9360000000000001e-05 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681012+0000", 2024-05-09T08:57:13.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107077 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681023+0000", 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1236000000000001e-05 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116939 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681272+0000", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013183499999999999 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681397+0000", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124456 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685651+0000", 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042543770000000002 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686328+0000", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067728299999999999 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686380+0000", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2012000000000003e-05 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686425+0000", 2024-05-09T08:57:13.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4305999999999999e-05 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070530598000005, 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060063249999999999, 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680910+0000", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680914+0000", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7799999999999998e-06 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681059+0000", 2024-05-09T08:57:13.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000144986 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681066+0000", 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4910000000000002e-06 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682105+0000", 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010389920000000001 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682185+0000", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0357000000000003e-05 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682323+0000", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828000000000001 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685656+0000", 2024-05-09T08:57:13.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033325149999999999 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686805+0000", 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001149529 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686877+0000", 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1303000000000005e-05 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686916+0000", 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9461e-05 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070524821999996, 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006182725, 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680916+0000", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.96e-07 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.61e-06 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:13.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00020153400000000001 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681129+0000", 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2620000000000004e-06 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682565+0000", 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014361650000000001 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682695+0000", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012924400000000001 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682808+0000", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113004 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685660+0000", 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002852121 2024-05-09T08:57:13.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686999+0000", 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001338802 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687050+0000", 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1625000000000001e-05 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687098+0000", 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8062000000000002e-05 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070520954000003, 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063447829999999997, 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680919+0000", 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680920+0000", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9200000000000002e-07 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680945+0000", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4856000000000001e-05 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681153+0000", 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000208385 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681161+0000", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3189999999999995e-06 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683100+0000", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019385590000000001 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683175+0000", 2024-05-09T08:57:13.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4972e-05 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683297+0000", 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012193699999999999 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685664+0000", 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023667929999999998 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687171+0000", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001507216 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687223+0000", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2571999999999997e-05 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687264+0000", 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0881999999999999e-05 2024-05-09T08:57:13.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070493788999997, 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064597120000000003, 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680947+0000", 2024-05-09T08:57:13.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1600000000000001e-07 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680950+0000", 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:13.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226572 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681193+0000", 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6884e-05 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683542+0000", 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023484650000000001 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683657+0000", 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115277 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683766+0000", 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010891399999999999 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685668+0000", 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001901743 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687313+0000", 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0016458709999999999 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687362+0000", 2024-05-09T08:57:13.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8625999999999998e-05 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687406+0000", 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4276000000000001e-05 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.782 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070489327000004, 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066291869999999999, 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680951+0000", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4799999999999999e-07 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680954+0000", 2024-05-09T08:57:13.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6730000000000001e-06 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681306+0000", 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000351491 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681313+0000", 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9619999999999996e-06 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684111+0000", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027984569999999999 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684202+0000", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1195999999999999e-05 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684353+0000", 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015071299999999999 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686447+0000", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020943440000000002 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687488+0000", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001040336 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687534+0000", 2024-05-09T08:57:13.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6774999999999999e-05 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687580+0000", 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5892e-05 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:13.788 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070485231999996, 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006791755, 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680955+0000", 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680956+0000", 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.34e-07 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680958+0000", 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-06 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681332+0000", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00037391099999999999 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681338+0000", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2639999999999999e-06 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684606+0000", 2024-05-09T08:57:13.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032684599999999999 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684723+0000", 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011661499999999999 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684853+0000", 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013001199999999999 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686475+0000", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001621883 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687653+0000", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001178438 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687705+0000", 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1805000000000003e-05 2024-05-09T08:57:13.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687747+0000", 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2143e-05 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070480877999998, 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069219210000000002, 2024-05-09T08:57:13.794 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:13.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680960+0000", 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6399999999999998e-07 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680968+0000", 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3289999999999994e-06 2024-05-09T08:57:13.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681351+0000", 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00038298100000000001 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681356+0000", 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.101e-06 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685141+0000", 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003785086 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685240+0000", 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8267e-05 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685406+0000", 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000166644 2024-05-09T08:57:13.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686653+0000", 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012463190000000001 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687796+0000", 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011430450000000001 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687849+0000", 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2923999999999997e-05 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687881+0000", 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2861000000000003e-05 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:13.800 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.070456406999995, 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048537759999999998, 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680984+0000", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680987+0000", 2024-05-09T08:57:13.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4370000000000001e-06 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681414+0000", 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00042719999999999998 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681421+0000", 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.601e-06 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682060+0000", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00063917100000000001 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682139+0000", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8872e-05 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682269+0000", 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013019400000000001 2024-05-09T08:57:13.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684249+0000", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001979794 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685731+0000", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014824930000000001 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685788+0000", 2024-05-09T08:57:13.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6986e-05 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685838+0000", 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9755000000000001e-05 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:20 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:13.806 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.027483576999998, 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012308524, 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723957+0000", 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723958+0000", 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.296e-06 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.395e-06 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723990+0000", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6914999999999999e-05 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723999+0000", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5129999999999996e-06 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724055+0000", 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6159000000000002e-05 2024-05-09T08:57:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724166+0000", 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110532 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.736265+0000", 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012099713999999999 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:15 2.3 e20/17 2:c27f454c:::mds1_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.025458149999999, 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061558979999999999, 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725982+0000", 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725983+0000", 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8000000000000004e-07 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725991+0000", 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0860000000000002e-06 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726006+0000", 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4938e-05 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2429999999999999e-06 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727846+0000", 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018329609999999999 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727871+0000", 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4406999999999999e-05 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732095+0000", 2024-05-09T08:57:13.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042245640000000001 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732138+0000", 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818999999999998e-05 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:12 2.c 2:36924187:::618.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021377413000003, 2024-05-09T08:57:13.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057959939999999996, 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730063+0000", 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1199999999999998e-07 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730066+0000", 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.694e-06 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730250+0000", 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018380499999999999 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730260+0000", 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7100000000000002e-06 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731230+0000", 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097010499999999997 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731303+0000", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3579999999999997e-05 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731420+0000", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116568 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732988+0000", 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015678440000000001 2024-05-09T08:57:13.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735786+0000", 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027978769999999998 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1066000000000003e-05 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735859+0000", 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2332999999999997e-05 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:13 2.1 2:81735765:::619.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021373015000002, 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048474440000000002, 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730067+0000", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730068+0000", 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730070+0000", 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.553e-06 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730295+0000", 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00022485300000000001 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730301+0000", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9580000000000004e-06 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730352+0000", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1400999999999998e-05 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730483+0000", 2024-05-09T08:57:13.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013079300000000001 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010618799999999999 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733630+0000", 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030401299999999998 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734823+0000", 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011937950000000001 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734873+0000", 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9707999999999999e-05 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734915+0000", 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1746999999999999e-05 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021368993000003, 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050848680000000002, 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.828 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730071+0000", 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730072+0000", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6100000000000002e-07 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730075+0000", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8490000000000002e-06 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730966+0000", 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089175099999999998 2024-05-09T08:57:13.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730979+0000", 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2847000000000001e-05 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731802+0000", 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082315399999999999 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731879+0000", 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7046999999999996e-05 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732012+0000", 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000133005 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733927+0000", 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001914612 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735049+0000", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011219789999999999 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735113+0000", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4362999999999993e-05 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735156+0000", 2024-05-09T08:57:13.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2899999999999999e-05 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021361552000002, 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.004901702, 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730079+0000", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1800000000000002e-07 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730081+0000", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8220000000000001e-06 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731051+0000", 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096957499999999997 2024-05-09T08:57:13.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731057+0000", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9560000000000002e-06 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731629+0000", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057226399999999995 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731714+0000", 2024-05-09T08:57:13.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5661999999999994e-05 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732235+0000", 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052096200000000003 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733613+0000", 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013775339999999999 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734929+0000", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001316198 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734954+0000", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4477999999999998e-05 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734981+0000", 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6933000000000001e-05 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.839 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021358315999997, 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053901399999999999, 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730082+0000", 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8700000000000002e-07 2024-05-09T08:57:13.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730085+0000", 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7609999999999999e-06 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731070+0000", 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098512399999999998 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731083+0000", 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3033999999999999e-05 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732979+0000", 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001895276 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733089+0000", 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110542 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733211+0000", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121471 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734283+0000", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001071904 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735386+0000", 2024-05-09T08:57:13.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011033620000000001 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735437+0000", 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0735999999999997e-05 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735472+0000", 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5642999999999997e-05 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021354207999998, 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054919699999999997, 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730086+0000", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730087+0000", 2024-05-09T08:57:13.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7599999999999997e-07 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730089+0000", 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731097+0000", 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001008412 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731102+0000", 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5589999999999999e-06 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732441+0000", 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00133946 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732543+0000", 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010173199999999999 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732613+0000", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9788000000000003e-05 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734007+0000", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013943499999999999 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735476+0000", 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014689709999999999 2024-05-09T08:57:13.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735536+0000", 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0408999999999997e-05 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735578+0000", 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1897000000000002e-05 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:23 2.c 2:3631e767:::102.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021350615999999, 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059167780000000001, 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:13.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730090+0000", 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5699999999999999e-07 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730096+0000", 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0599999999999996e-06 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731114+0000", 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001017525 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731118+0000", 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7639999999999996e-06 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731633+0000", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00051442600000000001 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731741+0000", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010863599999999999 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731906+0000", 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016487199999999999 2024-05-09T08:57:13.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733654+0000", 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001747738 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735909+0000", 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022545 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735950+0000", 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1201000000000001e-05 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.736007+0000", 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6799000000000003e-05 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021336753999996, 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055945150000000004, 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730104+0000", 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000002e-07 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730113+0000", 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1020000000000003e-06 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731131+0000", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010174279999999999 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731135+0000", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4449999999999998e-06 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733092+0000", 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019572190000000001 2024-05-09T08:57:13.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733229+0000", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013633499999999999 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733345+0000", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116647 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734293+0000", 2024-05-09T08:57:13.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094760700000000005 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735618+0000", 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013248260000000001 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735667+0000", 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9468000000000001e-05 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735698+0000", 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1053000000000002e-05 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.021325930000003, 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057439600000000002, 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:13.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730114+0000", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730115+0000", 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730117+0000", 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7940000000000001e-06 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731147+0000", 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010300610000000001 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731151+0000", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7269999999999998e-06 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733642+0000", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024900780000000002 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733758+0000", 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116214 2024-05-09T08:57:13.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735858+0000", 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021007040000000001 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:13.867 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:13.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:13.869 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:13.869+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:13.970 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:13.971+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:14.023 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:14.023 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:14.033 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:14.070 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:14.071+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:14.148 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:14.149 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:14.149 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:14.149 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:14.149 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:14.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:14.171 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:14.172+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:14.272 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:14.273+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:14.283 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:14.283 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:14.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:14.284 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:14.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.284 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:14.284 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.601743627000005, 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:14.285 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:14.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:14.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:14.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:14.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.294 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:14.294 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:14.294 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.601723656000004, 2024-05-09T08:57:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:14.295 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:14.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:14.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:14.301 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:14.302 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.600894890000006, 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:14.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:14.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:14.309 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.599891998000004, 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:14.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:14.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.599589167000005, 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:14.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:14.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.558402608999998, 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:14.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:14.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:14.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:14.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.558390582999998, 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:14.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:14.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.558377112999999, 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.336 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.557352711, 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:14.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:14.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:14.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.556925421999999, 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:14.342 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.347 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.556809082000001, 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.555535085999999, 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:14.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.356 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.553785685999998, 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:14.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.553457541999997, 2024-05-09T08:57:14.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:14.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:14.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 62.543328914, 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:14.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:14.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.552585913000001, 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.373 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:14.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:14.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:14.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.380 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.552576526999999, 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:14.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:14.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:14.387 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.552565483000002, 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:14.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:14.392 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 56.550830486999999, 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:14.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:14.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:14.398 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 44.520926281000001, 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:14.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:14.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:14.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:14.405 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:14.405 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:14.405 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:57:14.405 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:14.406 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:57:14.406 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T08:57:14.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:57:14.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:57:14.407 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:14.407+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:14.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:14.435 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T08:57:14.435 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-05-09T08:57:14.435 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:57:14.435 DEBUG:teuthology.orchestra.run.smithi049:> type ceph-objectstore-tool 2024-05-09T08:57:14.498 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-09T08:57:14.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-05-09T08:57:14.502 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:14.506 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:14.508+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:14.520 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:14.520 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:14.520 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:14.520 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:14.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.544+0000 7fbb97ba9640 1 Processor -- start 2024-05-09T08:57:14.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.544+0000 7fbb97ba9640 1 -- start start 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb97ba9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb90154d50 0x7fbb900707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb97ba9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb97ba9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015b940 con 0x7fbb9006fa70 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb95945640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb90154d50 0x7fbb900707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb97ba9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9015bc20 con 0x7fbb90154d50 2024-05-09T08:57:14.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb97ba9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900c8e30 con 0x7fbb9006f630 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb95945640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbb90154d50 0x7fbb900707d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38162/0 (socket says 172.21.15.49:38162) 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.545+0000 7fbb95945640 1 -- 172.21.15.49:0/323983961 learned_addr learned my addr 172.21.15.49:0/323983961 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb95945640 1 -- 172.21.15.49:0/323983961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 msgr2=0x7fbb9015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb95144640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb95945640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb95945640 1 -- 172.21.15.49:0/323983961 >> v1:172.21.15.49:6789/0 conn(0x7fbb9006fa70 legacy=0x7fbb9015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb95945640 1 -- 172.21.15.49:0/323983961 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb900c9110 con 0x7fbb90154d50 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.546+0000 7fbb94943640 1 -- 172.21.15.49:0/323983961 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015026588 0 0) 0x7fbb9015b940 con 0x7fbb9006fa70 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.547+0000 7fbb95945640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90154d50 0x7fbb900707d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009870 tx=0x7fbb8c02a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea6e257c213ea50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.547+0000 7fbb94943640 1 -- 172.21.15.49:0/323983961 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c037070 con 0x7fbb90154d50 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.547+0000 7fbb94943640 1 -- 172.21.15.49:0/323983961 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8c032640 con 0x7fbb90154d50 2024-05-09T08:57:14.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.547+0000 7fbb94943640 1 -- 172.21.15.49:0/323983961 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c032a10 con 0x7fbb90154d50 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 -- 172.21.15.49:0/323983961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90154d50 msgr2=0x7fbb900707d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90154d50 0x7fbb900707d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009870 tx=0x7fbb8c02a2f0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 -- 172.21.15.49:0/323983961 shutdown_connections 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/323983961 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb90154d50 0x7fbb900707d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 -- 172.21.15.49:0/323983961 >> 172.21.15.49:0/323983961 conn(0x7fbb900ce390 msgr2=0x7fbb900ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 -- 172.21.15.49:0/323983961 shutdown_connections 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.548+0000 7fbb97ba9640 1 -- 172.21.15.49:0/323983961 wait complete. 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.549+0000 7fbb97ba9640 1 Processor -- start 2024-05-09T08:57:14.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.549+0000 7fbb97ba9640 1 -- start start 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.550+0000 7fbb97ba9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.550+0000 7fbb97ba9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb9006fa70 0x7fbb901f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.550+0000 7fbb97ba9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb97ba9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9006d220 con 0x7fbb901f1370 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb97ba9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb90155130 con 0x7fbb9006fa70 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb97ba9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9006b930 con 0x7fbb9006f630 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38172/0 (socket says 172.21.15.49:38172) 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 -- 172.21.15.49:0/3610725027 learned_addr learned my addr 172.21.15.49:0/3610725027 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb95144640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb9006fa70 0x7fbb901f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb95945640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 -- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 msgr2=0x7fbb9015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 -- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb9006fa70 msgr2=0x7fbb901f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb9006fa70 0x7fbb901f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 -- 172.21.15.49:0/3610725027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb901f1820 con 0x7fbb901f1370 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb95945640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.551+0000 7fbb96146640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb88001560 tx=0x7fbb8800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.552+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88017070 con 0x7fbb901f1370 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.552+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88003dc0 con 0x7fbb901f1370 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.552+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880052b0 con 0x7fbb901f1370 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.552+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9015dfb0 con 0x7fbb901f1370 2024-05-09T08:57:14.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.552+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9015c080 con 0x7fbb901f1370 2024-05-09T08:57:14.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.553+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9004f660 con 0x7fbb901f1370 2024-05-09T08:57:14.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.554+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb8801a070 con 0x7fbb901f1370 2024-05-09T08:57:14.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.554+0000 7fbb867fc640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 0x7fbb7406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.554+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5857+0+0 (secure 0 0 0) 0x7fbb8808a440 con 0x7fbb901f1370 2024-05-09T08:57:14.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.554+0000 7fbb95945640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 0x7fbb7406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.555+0000 7fbb95945640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 0x7fbb7406e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.560+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb88058750 con 0x7fbb901f1370 2024-05-09T08:57:14.607 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:14.608+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:14.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.657+0000 7fe4e4d34640 1 Processor -- start 2024-05-09T08:57:14.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.657+0000 7fe4e4d34640 1 -- start start 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4e4d34640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe4e0154b90 0x7fe4e00cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4e4d34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4e4d34640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e0072600 con 0x7fe4e00c6640 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4e4d34640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e00728e0 con 0x7fe4e0154b90 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4e4d34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e00c5880 con 0x7fe4e0154f70 2024-05-09T08:57:14.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4de59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe4e0154b90 0x7fe4e00cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4de59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe4e0154b90 0x7fe4e00cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38174/0 (socket says 172.21.15.49:38174) 2024-05-09T08:57:14.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4de59c640 1 -- 172.21.15.49:0/1457706112 learned_addr learned my addr 172.21.15.49:0/1457706112 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:14.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4ddd9b640 1 -- 172.21.15.49:0/1457706112 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210047790 0 0) 0x7fe4e0072600 con 0x7fe4e00c6640 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.658+0000 7fe4ddd9b640 1 -- 172.21.15.49:0/1457706112 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4cc003520 con 0x7fe4e00c6640 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4de59c640 1 -- 172.21.15.49:0/1457706112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 msgr2=0x7fe4e00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4de59c640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e00cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4de59c640 1 -- 172.21.15.49:0/1457706112 >> v1:172.21.15.183:6789/0 conn(0x7fe4e00c6640 legacy=0x7fe4e00c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4de59c640 1 -- 172.21.15.49:0/1457706112 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e00c5b60 con 0x7fe4e0154b90 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:14.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.659+0000 7fe4de59c640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e00cb280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009870 tx=0x7fe4d002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5c8717e9e3ca8a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.660+0000 7fe4ddd9b640 1 -- 172.21.15.49:0/1457706112 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0037070 con 0x7fe4e0154b90 2024-05-09T08:57:14.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.660+0000 7fe4ddd9b640 1 -- 172.21.15.49:0/1457706112 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d002ae50 con 0x7fe4e0154b90 2024-05-09T08:57:14.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.660+0000 7fe4ddd9b640 1 -- 172.21.15.49:0/1457706112 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0032860 con 0x7fe4e0154b90 2024-05-09T08:57:14.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.660+0000 7fe4e4d34640 1 -- 172.21.15.49:0/1457706112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 msgr2=0x7fe4e00cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.660+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e00cb280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009870 tx=0x7fe4d002a300 comp rx=0 tx=0).stop 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 -- 172.21.15.49:0/1457706112 shutdown_connections 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/1457706112 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e00cb280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 -- 172.21.15.49:0/1457706112 >> 172.21.15.49:0/1457706112 conn(0x7fe4e0074840 msgr2=0x7fe4e0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 -- 172.21.15.49:0/1457706112 shutdown_connections 2024-05-09T08:57:14.661 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.661+0000 7fe4e4d34640 1 -- 172.21.15.49:0/1457706112 wait complete. 2024-05-09T08:57:14.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.662+0000 7fe4e4d34640 1 Processor -- start 2024-05-09T08:57:14.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.662+0000 7fe4e4d34640 1 -- start start 2024-05-09T08:57:14.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e00c6640 0x7fe4e0159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e015af40 con 0x7fe4e0154b90 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e00c6120 con 0x7fe4e00c6640 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4e4d34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e0071820 con 0x7fe4e0154f70 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4de59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e00c6640 0x7fe4e0159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e0157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38182/0 (socket says 172.21.15.49:38182) 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.663+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 learned_addr learned my addr 172.21.15.49:0/4058308711 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 msgr2=0x7fe4e0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe4e0154f70 0x7fe4e0157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e00c6640 msgr2=0x7fe4e0159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e00c6640 0x7fe4e0159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01f4a90 con 0x7fe4e0154b90 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4e00c6640 0x7fe4e0159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe4e0154b90 0x7fe4e0157110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800e960 tx=0x7fe4c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8017070 con 0x7fe4e0154b90 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c800ca00 con 0x7fe4e0154b90 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800cdd0 con 0x7fe4e0154b90 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e0148920 con 0x7fe4e0154b90 2024-05-09T08:57:14.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.664+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e0147b70 con 0x7fe4e0154b90 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.665+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe4e0069060 con 0x7fe4e0154b90 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.666+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4c801a070 con 0x7fe4e0154b90 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4d6ffd640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe4c006be60 0x7fe4c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5857+0+0 (secure 0 0 0) 0x7fe4c808b360 con 0x7fe4e0154b90 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe4c006be60 0x7fe4c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v38) ==== 54+0+16 (secure 0 0 0) 0x7fe4e0069060 con 0x7fe4e0154b90 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe4e0159680 0x7fe4e00b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.667+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4e01f4d50 con 0x7fe4e0159680 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.668+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe4e0159680 0x7fe4e00b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.668+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe4c006be60 0x7fe4c006e320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0034000 tx=0x7fe4d002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.668+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe4e0159680 0x7fe4e00b4660 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.669+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e01f4d50 con 0x7fe4e0159680 2024-05-09T08:57:14.673 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:14.673 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:14.673 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:14.673 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:14.673 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:14.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:14.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.684+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e004f660 con 0x7fe4e0159680 2024-05-09T08:57:14.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.685+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e004f660 con 0x7fe4e0159680 2024-05-09T08:57:14.684 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:14.685 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:14.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.685+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe4e01f3e40 0x7fe4e01f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.685+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe4e00cbde0 con 0x7fe4e01f3e40 2024-05-09T08:57:14.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.685+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe4e01f3e40 0x7fe4e01f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.686+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe4e01f3e40 0x7fe4e01f4280 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:14.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.686+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e00cbde0 con 0x7fe4e01f3e40 2024-05-09T08:57:14.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.702+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e01f47c0 con 0x7fe4e01f3e40 2024-05-09T08:57:14.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.703+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e01f47c0 con 0x7fe4e01f3e40 2024-05-09T08:57:14.702 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:14.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:14.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.703+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:14.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.703+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe4e01f1720 con 0x7fe4e01f0de0 2024-05-09T08:57:14.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.704+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6808/136930210 2024-05-09T08:57:14.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.704+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:57:14.728 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:14.729+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:14.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.771+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fbb900731a0 con 0x7fbb901f1370 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:14.787 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.062107709999999, 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:14.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.062087110999997, 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:14.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:14.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.062082240000002, 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.800 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:14.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.804 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.058523940999997, 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:14.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.058515712999998, 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.058511904, 2024-05-09T08:57:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.055955875999999, 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.041455069000001, 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.041439597999997, 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.041434854000002, 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.041428437, 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025452217999998, 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025439515999999, 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025431599000001, 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025429090999999, 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025426627000002, 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025411331999997, 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025407010999999, 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.025403871000002, 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.022944070999998, 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:14.900 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:14.902 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:14.902 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:14.903+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.905+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6808/136930210 2024-05-09T08:57:14.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:14.905+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T08:57:15.003 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:15.004+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:15.046 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.046 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:15.046 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:15.046 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:15.104 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:15.105+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:15.157 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.157 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:15.157 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:15.157 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:15.157 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:15.205 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:15.206+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.589138886000001, 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:15.270 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:15.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.589136018999994, 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.276 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:15.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.589132374000002, 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:15.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:15.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.546323209000001, 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.287 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:15.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:15.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.541531220000003, 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.293 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:15.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.539024851999997, 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:15.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.537553461999998, 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.303 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:15.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.537477623999997, 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:15.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:15.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524656213999997, 2024-05-09T08:57:15.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:15.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:15.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524635216, 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.318 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:15.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524630821999999, 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.324 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:15.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:15.329 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524625764, 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.330 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:15.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:15.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:15.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524622282999999, 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:15.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:15.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:15.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524617665000001, 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:15.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:15.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:15.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:15.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:15.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524614196000002, 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:15.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524610895000002, 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.355 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:15.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:15.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.524297826000002, 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:15.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:15.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:15.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:15.365 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.508222003999997, 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:15.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:15.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.508217324999997, 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.371 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:15.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:15.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.508211307000003, 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:15.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:15.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:15.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.382 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.306+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6808/136930210 2024-05-09T08:57:15.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.306+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T08:57:15.384 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:15.385+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:15.385 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.397+0000 7fbb867fc640 1 -- 172.21.15.49:0/3610725027 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v39) ==== 101+0+0 (secure 0 0 0) 0x7fbb8805ca50 con 0x7fbb901f1370 2024-05-09T08:57:15.396 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:57:15.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 msgr2=0x7fbb7406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 0x7fbb7406e370 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c034000 tx=0x7fbb8c02e000 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 msgr2=0x7fbb90149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbb88001560 tx=0x7fbb8800cc70 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 shutdown_connections 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbb7406beb0 0x7fbb7406e370 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbb901f1370 0x7fbb90149c10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb9006fa70 0x7fbb901f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 --2- 172.21.15.49:0/3610725027 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbb9006f630 0x7fbb9015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:15.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.400+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 >> 172.21.15.49:0/3610725027 conn(0x7fbb900ce390 msgr2=0x7fbb900cae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.401+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 shutdown_connections 2024-05-09T08:57:15.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:15.401+0000 7fbb97ba9640 1 -- 172.21.15.49:0/3610725027 wait complete. 2024-05-09T08:57:15.485 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:15.486+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:15.489 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.490 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:15.490 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:15.490 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.499 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:15.586 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:15.587+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:15.602 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.602 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:15.602 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:15.602 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:15.603 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.611 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:15.689 INFO:teuthology.orchestra.run.smithi049.stdout:3.9 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.3d 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.3b 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.39 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.2e 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.3c 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.2d 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.21 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.1d 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:2.e 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:3.4 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:2.c 2024-05-09T08:57:15.690 INFO:teuthology.orchestra.run.smithi049.stdout:2.3 2024-05-09T08:57:15.691 INFO:teuthology.orchestra.run.smithi049.stdout:3.0 2024-05-09T08:57:15.691 INFO:teuthology.orchestra.run.smithi049.stdout:2.1 2024-05-09T08:57:15.691 INFO:teuthology.orchestra.run.smithi049.stdout:2.10 2024-05-09T08:57:15.696 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:15.697+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:15.757 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.093995015999994, 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.758 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:15.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:15.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:15.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701338751999998, 2024-05-09T08:57:15.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:15.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:15.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.771 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701291159999997, 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:15.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:15.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701241292999995, 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.778 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:15.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:15.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:15.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701235492999999, 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:15.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:15.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701123693, 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.790 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:15.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701026530999997, 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.796 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:15.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:15.801 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.701020772999996, 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.802 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:15.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:15.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.700947133, 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.808 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:15.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:15.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.700895321000004, 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:15.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:15.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.700891601999999, 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.820 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:15.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.215835145, 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.826 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:15.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.215818661, 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:15.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:15.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.215736703999994, 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:15.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.215689143999995, 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:15.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:15.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.012060699999999, 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:15.850 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:15.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:15.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.995981254999997, 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:15.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:15.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:15.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.861 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.995976939000002, 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:15.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:15.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.995967976000003, 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.867 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:15.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:15.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.995963916000001, 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:15.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:15.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:15.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:15.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:15.878 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:15.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:15.880 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:15.880+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:15.980 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:15.982+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:15.990 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:15.990 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:15.999 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:16.081 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:16.082+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:16.107+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6808/136930210 2024-05-09T08:57:16.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:16.107+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T08:57:16.110 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:16.110 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:16.119 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:16.181 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:16.182+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:16.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 2.10 --file /var/log/ceph/exp.2.10.2 2024-05-09T08:57:16.233 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:16.233 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:16.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:16.282 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:16.284+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:16.384 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:16.385+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:16.395 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:16.395 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:16.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:16.485 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:16.486+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:16.520 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:16.520 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:16.529 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:16.586 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:16.587+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:16.687 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:16.688+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:16.691 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:16.691 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:16.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:16.787 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:16.789+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:16.815 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:16.815 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:16.815 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:16.815 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:16.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:16.888 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:16.890+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:16.949 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:16.950 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:16.950 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:16.950 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:16.950 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:16.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:16.990 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:16.991+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:17.073 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:17.073 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:17.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:17.073 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 70.392169502000002, 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:17.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 70.392149531000001, 2024-05-09T08:57:17.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:17.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 70.391320765000003, 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 70.390317873000001, 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 70.390015042000002, 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.348828483999995, 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.348816458000002, 2024-05-09T08:57:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.348802988000003, 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.347778586000004, 2024-05-09T08:57:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.347351297000003, 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.347234956999998, 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.345960961000003, 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.344211560999995, 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.343883417000001, 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.333754788999997, 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:17.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:17.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.343011787999998, 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.176 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:17.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.184 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.343002402000003, 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:17.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:17.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.192 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.342991357999999, 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:17.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:17.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:17.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:17.200 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.341256362000003, 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.201 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:17.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:17.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:17.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.208 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.311352155999998, 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:17.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:17.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:17.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:17.216 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:17.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:17.217 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:17.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:17.219 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:17.219+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:17.319 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:17.320+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:17.324 INFO:teuthology.orchestra.run.smithi049.stderr:Exporting 2.10 info 2.10( v 20'17 (0'0,20'17] local-lis/les=17/18 n=8 ec=17/17 lis/c=17/17 les/c/f=18/18/0 sis=17) 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:091e33df:::502.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:09401a81:::601.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0bd6d154:::602.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0d82a743:::600.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0d89b25e:::603.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0daba267:::402.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0e9da45e:::61b.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Read #2:0f65c9b8:::60a.00000000:head# 2024-05-09T08:57:17.331 INFO:teuthology.orchestra.run.smithi049.stderr:Export successful 2024-05-09T08:57:17.333 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:17.333 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:17.333 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:17.333 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:17.342 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:17.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:57:17.427 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:17.428+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:17.488 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:17.488 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:17.488 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:17.488 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:17.488 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:17.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:17.520 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:17.521+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:17.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 Processor -- start 2024-05-09T08:57:17.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 -- start start 2024-05-09T08:57:17.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f24dc0c6b80 0x7f24dc071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24dc0c6fd0 con 0x7f24dc154b90 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f24dc072600 con 0x7f24dc0c6b80 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.525+0000 7f24e3031640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc0728e0 con 0x7f24dc154f70 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24e0dcd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f24dc154b90 0x7f24dc0cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59720/0 (socket says 172.21.15.49:59720) 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24e0dcd640 1 -- 172.21.15.49:0/3937793402 learned_addr learned my addr 172.21.15.49:0/3937793402 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24dbfff640 1 --2- 172.21.15.49:0/3937793402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24e15ce640 1 --2- 172.21.15.49:0/3937793402 >> v2:172.21.15.183:3300/0 conn(0x7f24dc0c6b80 0x7f24dc071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24db7fe640 1 -- 172.21.15.49:0/3937793402 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2171348744 0 0) 0x7f24dc0c6fd0 con 0x7f24dc154b90 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24db7fe640 1 -- 172.21.15.49:0/3937793402 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24cc003520 con 0x7f24dc154b90 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24dbfff640 1 -- 172.21.15.49:0/3937793402 >> v2:172.21.15.183:3300/0 conn(0x7f24dc0c6b80 msgr2=0x7f24dc071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:17.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24dbfff640 1 --2- 172.21.15.49:0/3937793402 >> v2:172.21.15.183:3300/0 conn(0x7f24dc0c6b80 0x7f24dc071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:17.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24dbfff640 1 -- 172.21.15.49:0/3937793402 >> v1:172.21.15.49:6789/0 conn(0x7f24dc154b90 legacy=0x7f24dc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:17.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.526+0000 7f24dbfff640 1 -- 172.21.15.49:0/3937793402 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc0c5880 con 0x7f24dc154f70 2024-05-09T08:57:17.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.527+0000 7f24dbfff640 1 --2- 172.21.15.49:0/3937793402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0c6640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f24c80097a0 tx=0x7f24c802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360e944296159d78 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:17.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.527+0000 7f24db7fe640 1 -- 172.21.15.49:0/3937793402 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8037070 con 0x7f24dc154f70 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.527+0000 7f24db7fe640 1 -- 172.21.15.49:0/3937793402 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c8032440 con 0x7f24dc154f70 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.527+0000 7f24db7fe640 1 -- 172.21.15.49:0/3937793402 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8032810 con 0x7f24dc154f70 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 -- 172.21.15.49:0/3937793402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 msgr2=0x7f24dc0c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 --2- 172.21.15.49:0/3937793402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0c6640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f24c80097a0 tx=0x7f24c802a210 comp rx=0 tx=0).stop 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 -- 172.21.15.49:0/3937793402 shutdown_connections 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 --2- 172.21.15.49:0/3937793402 >> v2:172.21.15.183:3300/0 conn(0x7f24dc0c6b80 0x7f24dc071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 --2- 172.21.15.49:0/3937793402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0c6640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 -- 172.21.15.49:0/3937793402 >> 172.21.15.49:0/3937793402 conn(0x7f24dc074840 msgr2=0x7f24dc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 -- 172.21.15.49:0/3937793402 shutdown_connections 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.528+0000 7f24e3031640 1 -- 172.21.15.49:0/3937793402 wait complete. 2024-05-09T08:57:17.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.529+0000 7f24e3031640 1 Processor -- start 2024-05-09T08:57:17.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.529+0000 7f24e3031640 1 -- start start 2024-05-09T08:57:17.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 0x7f24dc1f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0c6120 con 0x7f24dc0c6b80 2024-05-09T08:57:17.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0c5e40 con 0x7f24dc154b90 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e3031640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc15af40 con 0x7f24dc154f70 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38196/0 (socket says 172.21.15.49:38196) 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 -- 172.21.15.49:0/2600266457 learned_addr learned my addr 172.21.15.49:0/2600266457 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24dbfff640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 0x7f24dc1f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e15ce640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 -- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 msgr2=0x7f24dc0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 -- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 msgr2=0x7f24dc1f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 0x7f24dc1f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24e0dcd640 1 -- 172.21.15.49:0/2600266457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc149fb0 con 0x7f24dc0c6b80 2024-05-09T08:57:17.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.530+0000 7f24dbfff640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 0x7f24dc1f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24e0dcd640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c460 tx=0x7f24d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0017070 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d0005ce0 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc14a290 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0007290 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.531+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc14f9f0 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.532+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc04f660 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.532+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24d001a070 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.533+0000 7f24d97fa640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 0x7f24c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.533+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5857+0+0 (secure 0 0 0) 0x7f24d008a2d0 con 0x7f24dc0c6b80 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.533+0000 7f24dbfff640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 0x7f24c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:17.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.534+0000 7f24dbfff640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 0x7f24c006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24c8034000 tx=0x7f24c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:17.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.541+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24d0058be0 con 0x7f24dc0c6b80 2024-05-09T08:57:17.622 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:17.623+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.961064469999997, 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.957644708000004, 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.957418199000003, 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.957364905999995, 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.909041334999998, 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.908515459999997, 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.908268128000003, 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 59.902828925000001, 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.894869761000002, 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.894305582000001, 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.894218508999998, 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:17.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.707 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.894149450999997, 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.708 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:17.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.892791142, 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.713 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.717 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.892745894999997, 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:17.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.892490627999997, 2024-05-09T08:57:17.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:17.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 53.892451932, 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:17.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:17.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:17.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.879508874000003, 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:17.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:17.740 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.878869926, 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.741 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:17.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:17.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.745 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.878315981999997, 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:17.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:17.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:17.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:17.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.752 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 47.875461844, 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:17.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:17.756 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.757 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.757 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.757 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:17.757 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:17.757 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.708+0000 7fe4d7fff640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.49:6808/136930210 2024-05-09T08:57:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.708+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T08:57:17.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:17.747+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f24dc158920 con 0x7f24dc0c6b80 2024-05-09T08:57:17.759 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:17.759+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:17.759 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:17.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 2.10 2024-05-09T08:57:17.863 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:17.863 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:17.863 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:17.863 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:17.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:17.889 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:17.890+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:17.960 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:17.961+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:17.973 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:17.974 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:17.982 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:18.066 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:18.067+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.401923875999998, 2024-05-09T08:57:18.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:18.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:18.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.401903277000002, 2024-05-09T08:57:18.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:18.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:18.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:18.143 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.401898406000001, 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:18.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:18.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.398340106999996, 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:18.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:18.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:18.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:18.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:18.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:18.156 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.398331878999997, 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:18.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.398328070000005, 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:18.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:18.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.395772041999997, 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.171 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:18.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:18.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.381271235, 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.177 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:18.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:18.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.381255764000002, 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:18.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:18.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:18.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.381251020000001, 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:18.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:18.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:18.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:18.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:18.198 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.381244602999999, 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:18.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365268383999997, 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:18.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:18.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:18.211 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365255681999997, 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:18.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365247764999999, 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.217 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:18.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:18.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:18.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365245256999998, 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:18.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:18.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.230 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365242793, 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:18.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:18.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365227498000003, 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:18.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.244 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365223176999997, 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:18.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.365220037, 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.249 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:18.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.253 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.362760237000003, 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:18.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.259 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:18.261 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:18.261 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:18.261+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:18.361 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:18.363+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:18.404 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:18.404 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:18.404 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:18.404 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:18.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.406+0000 7f24d97fa640 1 -- 172.21.15.49:0/2600266457 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v41) ==== 87+0+0 (secure 0 0 0) 0x7f24d00584e0 con 0x7f24dc0c6b80 2024-05-09T08:57:18.405 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 msgr2=0x7f24c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 0x7f24c006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f24c8034000 tx=0x7f24c802e000 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 msgr2=0x7f24dc1f0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f24d000c460 tx=0x7f24d000c920 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 shutdown_connections 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f24c006be30 0x7f24c006e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24dc154f70 0x7f24dc0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc154b90 0x7f24dc1f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 --2- 172.21.15.49:0/2600266457 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f24dc0c6b80 0x7f24dc1f0870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 >> 172.21.15.49:0/2600266457 conn(0x7f24dc074840 msgr2=0x7f24dc0cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:18.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 shutdown_connections 2024-05-09T08:57:18.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:18.409+0000 7f24e3031640 1 -- 172.21.15.49:0/2600266457 wait complete. 2024-05-09T08:57:18.415 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:18.462 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:18.463+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:18.558 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:18.558 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:18.558 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:18.564 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:18.565+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:18.566 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:18.664 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:18.666+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:18.712 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:18.712 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.031878503000002, 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:18.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:18.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.031875635999995, 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:18.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:18.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:18.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.031871991000003, 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.728 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.989062825999994, 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:18.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:18.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.984270836999997, 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.734 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:18.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.739 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.981764468999998, 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:18.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:18.743 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.980293078999999, 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:18.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:18.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.980217240999998, 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:18.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:18.755 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967395830999997, 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:18.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:18.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967374833000001, 2024-05-09T08:57:18.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967370439, 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:18.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967365381, 2024-05-09T08:57:18.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:18.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:18.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:18.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:18.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.785 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.9673619, 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:18.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:18.791 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967357282000002, 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:18.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.797 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967353813000003, 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:18.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:18.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.803 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967350512000003, 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:18.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:18.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 54.967037443000002, 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:18.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.950961620999998, 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:18.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:18.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.950956941999998, 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:18.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.950950923999997, 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:18.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:18.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:18.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:18.832 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:18.832 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:18.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:18.834 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:18.834+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:18.933 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:18.934+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:18.977 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:18.977 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:18.977 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:18.977 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:18.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:18.987 INFO:teuthology.orchestra.run.smithi049.stdout: marking collection for removal 2024-05-09T08:57:18.987 INFO:teuthology.orchestra.run.smithi049.stdout:setting '_remove' omap key 2024-05-09T08:57:18.987 INFO:teuthology.orchestra.run.smithi049.stdout:finish_remove_pgs 2.10_head removing 2.10 2024-05-09T08:57:18.988 INFO:teuthology.orchestra.run.smithi049.stdout:Remove successful 2024-05-09T08:57:19.035 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:19.036+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:19.089 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:19.089 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:19.089 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:19.089 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:19.089 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:19.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:19.135 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:19.136+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:19.202 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:19.202 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:19.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.538726062999999, 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.146069799000003, 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.213 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:19.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.220 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.146022207000001, 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:19.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:19.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.226 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.14597234, 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.227 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:19.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:19.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.233 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145966540000003, 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.234 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:19.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145854740000004, 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:19.241 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:19.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:19.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:19.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145757578000001, 2024-05-09T08:57:19.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:19.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:19.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:19.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145751820000001, 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.257 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:19.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145678180000004, 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.264 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:19.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:19.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:19.271 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145626367999995, 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:19.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:19.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.145622649000003, 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.279 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:19.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.285 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.660566192000005, 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:19.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:19.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.292 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.660549708000005, 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.293 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:19.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:19.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.300 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.660467750999999, 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:19.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:19.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:19.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:19.307 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.660420191, 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.308 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:19.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:19.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 55.456791746999997, 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:19.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 49.440712302000001, 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.321 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:19.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:19.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:19.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:19.328 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 49.440707986, 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:19.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:19.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.335 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 49.440699023000001, 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:19.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 49.440694962999999, 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:19.342 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:19.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:19.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:19.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:19.348 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:19.348 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:19.350 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:19.350 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:19.350+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:19.453 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:19.453+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:19.464 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:19.464 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:19.473 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:19.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.10.2 2024-05-09T08:57:19.552 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:19.552+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:19.589 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:19.590 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:19.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d77fe640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=39}) -- 0x7fe4bc0034d0 con 0x7fe4e0154b90 2024-05-09T08:57:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d77fe640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] -- ping magic: 0 -- 0x7fe4bc0038a0 con 0x7fe4e01f0de0 2024-05-09T08:57:19.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 7 ==== osd_map(39..41 src has 1..41) ==== 1242+0+0 (secure 0 0 0) 0x7fe4c8089900 con 0x7fe4e0154b90 2024-05-09T08:57:19.665 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:19.665+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4e01f47c0 con 0x7fe4e0154b90 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 msgr2=0x7fe4e01f11e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d6ffd640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6808/136930210,v1:172.21.15.49:6809/136930210] conn(0x7fe4e01f0de0 0x7fe4e01f11e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.665+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== mon.0 v2:172.21.15.49:3300/0 8 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe4c8021020 con 0x7fe4e0154b90 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.666+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe4e0062180 0x7fe4e0062590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.666+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fe4e0074cc0 con 0x7fe4e0062180 2024-05-09T08:57:19.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.666+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe4e0062180 0x7fe4e0062590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.666+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe4e0062180 0x7fe4e0062590 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:19.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.667+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e0074cc0 con 0x7fe4e0062180 2024-05-09T08:57:19.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.681+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e00c6c10 con 0x7fe4e0062180 2024-05-09T08:57:19.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.683+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e00c6c10 con 0x7fe4e0062180 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.683+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe4e00b58d0 0x7fe4e002c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.683+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fe4e00b5d10 con 0x7fe4e00b58d0 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.684+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe4e00b58d0 0x7fe4e002c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:19.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.684+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe4e00b58d0 0x7fe4e002c040 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:19.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.685+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e00b5d10 con 0x7fe4e00b58d0 2024-05-09T08:57:19.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.699+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e002c850 con 0x7fe4e00b58d0 2024-05-09T08:57:19.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.701+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e002c850 con 0x7fe4e00b58d0 2024-05-09T08:57:19.700 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.701+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe4e002cce0 0x7fe4e002d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.701+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fe4e00b5f50 con 0x7fe4e002cce0 2024-05-09T08:57:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.702+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe4e002cce0 0x7fe4e002d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:19.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.702+0000 7fe4de59c640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe4e002cce0 0x7fe4e002d120 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:19.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.703+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e00b5f50 con 0x7fe4e002cce0 2024-05-09T08:57:19.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.719+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e00b5f50 con 0x7fe4e002cce0 2024-05-09T08:57:19.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.721+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e00b5f50 con 0x7fe4e002cce0 2024-05-09T08:57:19.720 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:19.721 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:19.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.721+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe4e02076e0 0x7fe4e0209ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:19.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.721+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fe4e020c970 con 0x7fe4e02076e0 2024-05-09T08:57:19.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.722+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe4e02076e0 0x7fe4e0209ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:19.721 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.722+0000 7fe4d7fff640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe4e02076e0 0x7fe4e0209ad0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:19.722 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.723+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e020c970 con 0x7fe4e02076e0 2024-05-09T08:57:19.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.737+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e020c970 con 0x7fe4e02076e0 2024-05-09T08:57:19.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.739+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e020c970 con 0x7fe4e02076e0 2024-05-09T08:57:19.738 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.739+0000 7fe4e4d34640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe4e020a520 0x7fe4e020a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.739+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fe4e020fd10 con 0x7fe4e020a520 2024-05-09T08:57:19.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.740+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe4e020a520 0x7fe4e020a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.740+0000 7fe4ded9d640 1 --2- 172.21.15.49:0/4058308711 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe4e020a520 0x7fe4e020a960 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:19.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.741+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4e020fd10 con 0x7fe4e020a520 2024-05-09T08:57:19.751 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:19.752+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:19.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.756+0000 7fe4e4d34640 1 -- 172.21.15.49:0/4058308711 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4e020af60 con 0x7fe4e020a520 2024-05-09T08:57:19.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:19.757+0000 7fe4d6ffd640 1 -- 172.21.15.49:0/4058308711 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4e020af60 con 0x7fe4e020a520 2024-05-09T08:57:19.757 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:19.757 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:19.758 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:19.758 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:19.760 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:19.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:19.852 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:19.853+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:19.926 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:19.927 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:19.937 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:19.978 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:19.979+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:20.054 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:20.055+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:20.090 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.090 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.099 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:20.154 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:20.155+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:20.210 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.210 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:20.260 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:20.261+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:20.357 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:20.358+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:20.381 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.381 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:20.420 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:57:20.457 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:20.459+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:20.504 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.504 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.514 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:20.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.543+0000 7f4c06c61640 1 Processor -- start 2024-05-09T08:57:20.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.543+0000 7f4c06c61640 1 -- start start 2024-05-09T08:57:20.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c06c61640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4c00151c00 0x7f4c000c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c06c61640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c000c9be0 con 0x7f4c000cd710 2024-05-09T08:57:20.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c06c61640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c000cdc50 con 0x7f4c000cd280 2024-05-09T08:57:20.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c06c61640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4c00157ac0 con 0x7f4c00151c00 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c051fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4c000cd710 0x7f4c0015b0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59748/0 (socket says 172.21.15.49:59748) 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c051fe640 1 -- 172.21.15.49:0/307836856 learned_addr learned my addr 172.21.15.49:0/307836856 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4c049fd640 1 --2- 172.21.15.49:0/307836856 >> v2:172.21.15.183:3300/0 conn(0x7f4c00151c00 0x7f4c000c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902467168 0 0) 0x7f4c000c9be0 con 0x7f4c000cd710 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.544+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf0003520 con 0x7f4c000cd710 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 216404648 0 0) 0x7f4c000cdc50 con 0x7f4c000cd280 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c000c9be0 con 0x7f4c000cd280 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3701857405 0 0) 0x7f4bf0003520 con 0x7f4c000cd710 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 >> v2:172.21.15.183:3300/0 conn(0x7f4c00151c00 msgr2=0x7f4c000c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 --2- 172.21.15.49:0/307836856 >> v2:172.21.15.183:3300/0 conn(0x7f4c00151c00 0x7f4c000c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 >> v1:172.21.15.183:6789/0 conn(0x7f4c000cd280 legacy=0x7f4c000c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c00157da0 con 0x7f4c000cd710 2024-05-09T08:57:20.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f4bf80019f0 con 0x7f4c000cd710 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 >> v1:172.21.15.49:6789/0 conn(0x7f4c000cd710 legacy=0x7f4c0015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4bf0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00157ac0 con 0x7f4bf0003b90 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf000bfe0 con 0x7f4bf0004800 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf000c1e0 con 0x7f4bf00051c0 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4bf0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4bfffff640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c051fe640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 -- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf00051c0 msgr2=0x7f4bf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:20.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf00051c0 0x7f4bf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 -- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf0004800 msgr2=0x7f4bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c049fd640 1 -- 172.21.15.49:0/307836856 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf0003520 con 0x7f4bf0003b90 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.545+0000 7f4c051fe640 1 -- 172.21.15.49:0/307836856 reap_dead start 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.546+0000 7f4c049fd640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4bf0003f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009ac0 tx=0x7f4bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9687ab76f0b978 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.546+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4036070 con 0x7f4bf0003b90 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.546+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf4031400 con 0x7f4bf0003b90 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.546+0000 7f4bff7fe640 1 -- 172.21.15.49:0/307836856 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4009690 con 0x7f4bf0003b90 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 -- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 msgr2=0x7f4bf0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4bf0003f90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009ac0 tx=0x7f4bf0005830 comp rx=0 tx=0).stop 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 -- 172.21.15.49:0/307836856 shutdown_connections 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 --2- 172.21.15.49:0/307836856 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4bf0003f90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.549 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 -- 172.21.15.49:0/307836856 >> 172.21.15.49:0/307836856 conn(0x7f4c0006b200 msgr2=0x7f4c0006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 -- 172.21.15.49:0/307836856 shutdown_connections 2024-05-09T08:57:20.550 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.547+0000 7f4c06c61640 1 -- 172.21.15.49:0/307836856 wait complete. 2024-05-09T08:57:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.548+0000 7f4c06c61640 1 Processor -- start 2024-05-09T08:57:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.548+0000 7f4c06c61640 1 -- start start 2024-05-09T08:57:20.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 0x7f4c000cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c000c6000 0x7f4c000c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00158640 con 0x7f4bf0003b90 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00158360 con 0x7f4c001568d0 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c06c61640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4c00151fe0 con 0x7f4c000c6000 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c049fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c049fd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38208/0 (socket says 172.21.15.49:38208) 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4c049fd640 1 -- 172.21.15.49:0/599648903 learned_addr learned my addr 172.21.15.49:0/599648903 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.549+0000 7f4bfffff640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 0x7f4c000cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c051fe640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c000c6000 0x7f4c000c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 -- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c000c6000 msgr2=0x7f4c000c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:20.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c000c6000 0x7f4c000c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 -- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 msgr2=0x7f4c000cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 0x7f4c000cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 -- 172.21.15.49:0/599648903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c00154020 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4bfffff640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 0x7f4c000cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c049fd640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009be0 tx=0x7f4bf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf403f020 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf4037040 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4036070 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c000c69c0 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.550+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c00154570 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.551+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0014e780 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.552+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bf4042020 con 0x7f4bf0003b90 2024-05-09T08:57:20.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.552+0000 7f4bfd7fa640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 0x7f4be406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.552+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5857+0+0 (secure 0 0 0) 0x7f4bf40aac40 con 0x7f4bf0003b90 2024-05-09T08:57:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.552+0000 7f4bfffff640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 0x7f4be406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.553+0000 7f4bfffff640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 0x7f4be406e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c00074c20 tx=0x7f4bec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:20.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.557+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c0014e780 con 0x7f4bf0003b90 2024-05-09T08:57:20.559 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:20.560+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:20.632 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.632 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.642 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:20.669 INFO:teuthology.orchestra.run.smithi049.stdout:get_pg_num_history pg_num_history pg_num_history(e38 pg_nums {1={14=1},2={17=32},3={18=64}} deleted_pools ) 2024-05-09T08:57:20.669 INFO:teuthology.orchestra.run.smithi049.stdout:Importing pgid 2.10 2024-05-09T08:57:20.669 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:091e33df:::502.00000000:head# 2024-05-09T08:57:20.669 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.669 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:20.670+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:20.670 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:09401a81:::601.00000000:head# 2024-05-09T08:57:20.670 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.670 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0bd6d154:::602.00000000:head# 2024-05-09T08:57:20.670 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.670 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0d82a743:::600.00000000:head# 2024-05-09T08:57:20.671 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.671 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0d89b25e:::603.00000000:head# 2024-05-09T08:57:20.671 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.671 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0daba267:::402.00000000:head# 2024-05-09T08:57:20.671 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0e9da45e:::61b.00000000:head# 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:Write #2:0f65c9b8:::60a.00000000:head# 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:snapset 0=[]:{} 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:write_pg epoch 20 info 2.10( v 20'17 (0'0,20'17] local-lis/les=17/18 n=8 ec=17/17 lis/c=17/17 les/c/f=18/18/0 sis=17) 2024-05-09T08:57:20.672 INFO:teuthology.orchestra.run.smithi049.stdout:Import successful 2024-05-09T08:57:20.759 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.759 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.760 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:20.761+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:20.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:20.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:20.777+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f4c000751b0 con 0x7f4bf0003b90 2024-05-09T08:57:20.861 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:20.863+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:20.870 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:20.871 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:20.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:20.964 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:20.965+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:21.023 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.023 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.032 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:21.064 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:21.065+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:21.165 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:21.166+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:21.175 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.175 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.184 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:21.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f /var/log/ceph/exp.2.10.2 2024-05-09T08:57:21.295 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:21.296+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:21.328 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.328 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.336 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:21.380 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:21.381+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:21.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.420+0000 7f4bfd7fa640 1 -- 172.21.15.49:0/599648903 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v42) ==== 97+0+0 (secure 0 0 0) 0x7f4c000751b0 con 0x7f4bf0003b90 2024-05-09T08:57:21.420 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:57:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 msgr2=0x7f4be406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 0x7f4be406e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c00074c20 tx=0x7f4bec009040 comp rx=0 tx=0).stop 2024-05-09T08:57:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 msgr2=0x7f4c001522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009be0 tx=0x7f4bf4004730 comp rx=0 tx=0).stop 2024-05-09T08:57:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 shutdown_connections 2024-05-09T08:57:21.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4be406be30 0x7f4be406e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c000c6000 0x7f4c000c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c001568d0 0x7f4c000cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 --2- 172.21.15.49:0/599648903 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4bf0003b90 0x7f4c001522c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 >> 172.21.15.49:0/599648903 conn(0x7f4c0006b200 msgr2=0x7f4c000c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 shutdown_connections 2024-05-09T08:57:21.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.423+0000 7f4c06c61640 1 -- 172.21.15.49:0/599648903 wait complete. 2024-05-09T08:57:21.442 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.442 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.451 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:21.475 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:21.476+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:21.567 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:21.568+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:21.594 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.594 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.603 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:21.668 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:21.669+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:21.748 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.748 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.757 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:21.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:21.801 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:21.802+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:21.861 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:21.862 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:21.870 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:21.870 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:21.870+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:21.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.880+0000 7fcd4b46b640 1 Processor -- start 2024-05-09T08:57:21.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.880+0000 7fcd4b46b640 1 -- start start 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd4b46b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcd440c7130 0x7fcd44158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd4b46b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd44072670 0x7fcd440c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd4b46b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd44159b10 con 0x7fcd44152e30 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd4b46b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fcd44159270 con 0x7fcd440c7130 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd4b46b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcd44159550 con 0x7fcd44072670 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd48a06640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcd440c7130 0x7fcd44158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd49207640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcd44152e30 0x7fcd440c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59758/0 (socket says 172.21.15.49:59758) 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.881+0000 7fcd49207640 1 -- 172.21.15.49:0/3064563991 learned_addr learned my addr 172.21.15.49:0/3064563991 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd44072670 0x7fcd440c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd48a06640 1 -- 172.21.15.49:0/3064563991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd44072670 msgr2=0x7fcd440c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd44072670 0x7fcd440c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd48a06640 1 -- 172.21.15.49:0/3064563991 >> v1:172.21.15.49:6789/0 conn(0x7fcd44152e30 legacy=0x7fcd440c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd3bfff640 1 -- 172.21.15.49:0/3064563991 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713784667 0 0) 0x7fcd44159b10 con 0x7fcd44152e30 2024-05-09T08:57:21.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd48a06640 1 -- 172.21.15.49:0/3064563991 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd44159830 con 0x7fcd440c7130 2024-05-09T08:57:21.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.882+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd440c7130 0x7fcd44158be0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcd340097a0 tx=0x7fcd3402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977deae9399dcaef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd3bfff640 1 -- 172.21.15.49:0/3064563991 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34037070 con 0x7fcd440c7130 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd3bfff640 1 -- 172.21.15.49:0/3064563991 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3402ae50 con 0x7fcd440c7130 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd3bfff640 1 -- 172.21.15.49:0/3064563991 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34032660 con 0x7fcd440c7130 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3064563991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd440c7130 msgr2=0x7fcd44158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd440c7130 0x7fcd44158be0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcd340097a0 tx=0x7fcd3402a100 comp rx=0 tx=0).stop 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3064563991 shutdown_connections 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd44072670 0x7fcd440c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3064563991 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd440c7130 0x7fcd44158be0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3064563991 >> 172.21.15.49:0/3064563991 conn(0x7fcd440cc290 msgr2=0x7fcd440cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3064563991 shutdown_connections 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.883+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3064563991 wait complete. 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 Processor -- start 2024-05-09T08:57:21.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 -- start start 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd44072670 0x7fcd44156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd440c7130 0x7fcd441dec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd44152e30 0x7fcd441515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440c6620 con 0x7fcd44072670 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440c59f0 con 0x7fcd44152e30 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd4b46b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4406b4b0 con 0x7fcd440c7130 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd49207640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd44072670 0x7fcd44156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd49207640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd44072670 0x7fcd44156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:38216/0 (socket says 172.21.15.49:38216) 2024-05-09T08:57:21.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd48a06640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd440c7130 0x7fcd441dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.885+0000 7fcd49207640 1 -- 172.21.15.49:0/3307202009 learned_addr learned my addr 172.21.15.49:0/3307202009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd44152e30 0x7fcd441515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 -- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd440c7130 msgr2=0x7fcd441dec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd440c7130 0x7fcd441dec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 -- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd44152e30 msgr2=0x7fcd441515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd44152e30 0x7fcd441515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd441f19d0 con 0x7fcd44072670 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd44152e30 0x7fcd441515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcd44072670 0x7fcd44156790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c4d0 tx=0x7fcd4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40018070 con 0x7fcd44072670 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd40007500 con 0x7fcd44072670 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd441f1cb0 con 0x7fcd44072670 2024-05-09T08:57:21.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004a20 con 0x7fcd44072670 2024-05-09T08:57:21.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.886+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd441f2270 con 0x7fcd44072670 2024-05-09T08:57:21.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.887+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fcd4404f660 con 0x7fcd44072670 2024-05-09T08:57:21.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.888+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd4001b070 con 0x7fcd44072670 2024-05-09T08:57:21.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.888+0000 7fcd39ffb640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcd2806be30 0x7fcd2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.888+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5857+0+0 (secure 0 0 0) 0x7fcd4008a140 con 0x7fcd44072670 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.888+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7fcd4008c7c0 con 0x7fcd44072670 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcd2806be30 0x7fcd2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fcd44150580 0x7fcd441ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd440c3470 con 0x7fcd44150580 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fcd44150580 0x7fcd441ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcd2806be30 0x7fcd2806e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd34034000 tx=0x7fcd3402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.889+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fcd44150580 0x7fcd441ee430 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.890+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd440c3470 con 0x7fcd44150580 2024-05-09T08:57:21.904 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.905+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd4406bff0 con 0x7fcd44150580 2024-05-09T08:57:21.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.907+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd4406bff0 con 0x7fcd44150580 2024-05-09T08:57:21.906 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.907+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fcd44150960 0x7fcd4414b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.907+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fcd4414bc50 con 0x7fcd44150960 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.907+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fcd44150960 0x7fcd4414b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.907+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fcd44150960 0x7fcd4414b710 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.908+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd4414bc50 con 0x7fcd44150960 2024-05-09T08:57:21.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.923+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd4414c160 con 0x7fcd44150960 2024-05-09T08:57:21.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd4414c160 con 0x7fcd44150960 2024-05-09T08:57:21.924 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=43}) -- 0x7fcd441547f0 con 0x7fcd44072670 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd44154b90 con 0x7fcd44072670 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcd40091070 con 0x7fcd44072670 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:21.925 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fcd44155150 0x7fcd44062180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fcd440626c0 con 0x7fcd44155150 2024-05-09T08:57:21.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.925+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fcd44155150 0x7fcd44062180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.926+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fcd44155150 0x7fcd44062180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.926+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd440626c0 con 0x7fcd44155150 2024-05-09T08:57:21.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.942+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd44062bd0 con 0x7fcd44155150 2024-05-09T08:57:21.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.943+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd44062bd0 con 0x7fcd44155150 2024-05-09T08:57:21.943 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:21.943 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:21.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.944+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fcd440cae40 0x7fcd4402c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.944+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fcd440cb260 con 0x7fcd440cae40 2024-05-09T08:57:21.943 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.944+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fcd440cae40 0x7fcd4402c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.944+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fcd440cae40 0x7fcd4402c040 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.944 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.945+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd440cb260 con 0x7fcd440cae40 2024-05-09T08:57:21.959 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.960+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd4402c580 con 0x7fcd440cae40 2024-05-09T08:57:21.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.962+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd4402c580 con 0x7fcd440cae40 2024-05-09T08:57:21.962 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.962+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fcd4402cca0 0x7fcd4402d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.962+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fcd4402c580 con 0x7fcd4402cca0 2024-05-09T08:57:21.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.962+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fcd4402cca0 0x7fcd4402d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.963+0000 7fcd49207640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fcd4402cca0 0x7fcd4402d100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.964+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd4402c580 con 0x7fcd4402cca0 2024-05-09T08:57:21.970 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:21.971+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:21.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.978+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd4402c580 con 0x7fcd4402cca0 2024-05-09T08:57:21.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.980+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd4402c580 con 0x7fcd4402cca0 2024-05-09T08:57:21.979 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.980+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcd442031c0 0x7fcd442055b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.980+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fcd44208380 con 0x7fcd442031c0 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.980+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcd442031c0 0x7fcd442055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.980+0000 7fcd48a06640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fcd442031c0 0x7fcd442055b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.980 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.981+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd44208380 con 0x7fcd442031c0 2024-05-09T08:57:21.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.996+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd44205af0 con 0x7fcd442031c0 2024-05-09T08:57:21.997 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.998+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd44205af0 con 0x7fcd442031c0 2024-05-09T08:57:21.997 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.998+0000 7fcd4b46b640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fcd44206170 0x7fcd44208560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.998+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fcd4420b330 con 0x7fcd44206170 2024-05-09T08:57:21.998 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.998+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fcd44206170 0x7fcd44208560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:21.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.999+0000 7fcd49a08640 1 --2- 172.21.15.49:0/3307202009 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fcd44206170 0x7fcd44208560 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:21.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:21.999+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd4420b330 con 0x7fcd44206170 2024-05-09T08:57:22.013 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:22.013 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:22.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:22.015+0000 7fcd4b46b640 1 -- 172.21.15.49:0/3307202009 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcd44208b20 con 0x7fcd44206170 2024-05-09T08:57:22.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:22.016+0000 7fcd39ffb640 1 -- 172.21.15.49:0/3307202009 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcd44208b20 con 0x7fcd44206170 2024-05-09T08:57:22.016 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:22.016 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:22.019 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:22.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:22.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:22.071 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:22.072+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:22.164 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:22.164 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:22.164 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:22.171 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:22.172+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:22.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:22.272 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:22.274+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:22.318 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.654767749000001, 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.319 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:22.327 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.262111485000005, 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.328 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:22.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:22.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.262063893000004, 2024-05-09T08:57:22.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:22.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:22.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.262014026000003, 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.341 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:22.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:22.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.348 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.262008226000006, 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:22.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:22.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:22.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:22.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261896426000007, 2024-05-09T08:57:22.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:22.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:22.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:22.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:22.363 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261799264000004, 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:22.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:22.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:22.369 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261793506000004, 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:22.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261719866000007, 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:22.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261668053999998, 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.382 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:22.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.261664335000006, 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:22.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:22.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:22.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.776607877999993, 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:22.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:22.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.776591393999993, 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:22.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:22.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.776509437000001, 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.406 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:22.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.776461877000003, 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:22.412 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:22.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:22.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.572833433, 2024-05-09T08:57:22.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:22.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:22.423 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.556753987999997, 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:22.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.429 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.556749672000002, 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:22.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:22.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.435 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.556740709000003, 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:22.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:22.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 52.556736649000001, 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:22.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:22.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:22.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:22.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:22.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:22.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:22.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:22.449 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:22.450 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:22.451 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:22.451+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:22.551 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:22.552+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:22.566 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:22.566 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:22.566 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:22.566 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:22.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:22.652 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:22.653+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:22.690 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:22.691 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:22.700 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:22.753 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:22.754+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:22.818 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:22.818 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:22.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.138215880999994, 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:22.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.824 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.138210311999998, 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:22.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:22.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.138178624000005, 2024-05-09T08:57:22.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:22.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.138173937999994, 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:22.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.138170110000004, 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:22.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:22.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.137287244000007, 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:22.846 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:22.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:22.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 71.091714358999994, 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:22.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:22.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 71.091041583999996, 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:22.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.088320917000004, 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:22.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:22.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.867 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.088312876000003, 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:22.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.088308556000001, 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:22.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:22.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.088304250999997, 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:22.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:22.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.886 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.088301002999998, 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:22.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.890 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.073228630000003, 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:22.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.073217382999999, 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:22.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.902 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.073212961999999, 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:22.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.057108997, 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:22.908 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:22.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:22.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.057097665999997, 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.914 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:22.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:22.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.057089716, 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:22.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:22.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.925 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.057083837999997, 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:22.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:22.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:22.930 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:22.930 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:22.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:22.932 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:22.932+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:23.032 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:23.033+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:23.081 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.082 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:23.082 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:23.082 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:23.091 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:23.132 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:23.133+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:23.233 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:23.234+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:23.241 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.241 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:23.241 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:23.241 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:23.241 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:23.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:23.334 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:23.335+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:23.401 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.401 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:23.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.720889030999999, 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:23.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.720864679000002, 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:23.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.720858923999998, 2024-05-09T08:57:23.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:23.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:23.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 76.720843693000006, 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:23.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:23.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:23.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 71.677505707999998, 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:23.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.434 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 71.677497747000004, 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:23.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:23.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.440 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.670289718000006, 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:23.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:23.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:23.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.670278972999995, 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:23.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656304400000003, 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.457 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:23.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:23.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656294291000002, 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656291652999997, 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656276050999999, 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656273401999997, 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656259276, 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656256997, 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656249092000003, 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.656247571000002, 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.636171722, 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.636150262000001, 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 53.636143257000001, 2024-05-09T08:57:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.525 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:23.525 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:57:23.527 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:23.527 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:23.527+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:23.626 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:23.628+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.678+0000 7f7f41a38640 1 Processor -- start 2024-05-09T08:57:23.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.678+0000 7f7f41a38640 1 -- start start 2024-05-09T08:57:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f41a38640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7f3c0c6d50 0x7f7f3c071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f41a38640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c072600 con 0x7f7f3c154b90 2024-05-09T08:57:23.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f41a38640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0728e0 con 0x7f7f3c154f70 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f41a38640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3c0c5880 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3affd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7f3c154b90 0x7f7f3c0cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57716/0 (socket says 172.21.15.49:57716) 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3affd640 1 -- 172.21.15.49:0/3956368605 learned_addr learned my addr 172.21.15.49:0/3956368605 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3b7fe640 1 --2- 172.21.15.49:0/3956368605 >> v2:172.21.15.183:3300/0 conn(0x7f7f3c0c6d50 0x7f7f3c071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 886712157 0 0) 0x7f7f3c072600 con 0x7f7f3c154b90 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f24003520 con 0x7f7f3c154b90 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125218103 0 0) 0x7f7f24003520 con 0x7f7f3c154b90 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/3956368605 >> v1:172.21.15.183:6789/0 conn(0x7f7f3c154f70 legacy=0x7f7f3c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/3956368605 >> v1:172.21.15.49:6789/0 conn(0x7f7f3c154b90 legacy=0x7f7f3c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:23.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.679+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/3956368605 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c0c5b60 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.680+0000 7f7f3b7fe640 1 --2- 172.21.15.49:0/3956368605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c071820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009d90 tx=0x7f7f3402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51752c2c017aa2df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.680+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c001670 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.680+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34007540 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.680+0000 7f7f39ffb640 1 -- 172.21.15.49:0/3956368605 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340078f0 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 -- 172.21.15.49:0/3956368605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 msgr2=0x7f7f3c071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 --2- 172.21.15.49:0/3956368605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c071820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009d90 tx=0x7f7f3402a710 comp rx=0 tx=0).stop 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 -- 172.21.15.49:0/3956368605 shutdown_connections 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 --2- 172.21.15.49:0/3956368605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c071820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 -- 172.21.15.49:0/3956368605 >> 172.21.15.49:0/3956368605 conn(0x7f7f3c074840 msgr2=0x7f7f3c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 -- 172.21.15.49:0/3956368605 shutdown_connections 2024-05-09T08:57:23.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.681+0000 7f7f41a38640 1 -- 172.21.15.49:0/3956368605 wait complete. 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.682+0000 7f7f41a38640 1 Processor -- start 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 -- start start 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:23.682 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f3c154b90 0x7f7f3c14f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c15af40 con 0x7f7f3c154f70 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c0c6120 con 0x7f7f3c0c6d50 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f41a38640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c0cb990 con 0x7f7f3c154b90 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.683+0000 7f7f3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55254/0 (socket says 172.21.15.49:55254) 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/2607263973 learned_addr learned my addr 172.21.15.49:0/2607263973 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3affd640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3a7fc640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f3c154b90 0x7f7f3c14f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f3c154b90 msgr2=0x7f7f3c14f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f3c154b90 0x7f7f3c14f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 msgr2=0x7f7f3c14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:23.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 -- 172.21.15.49:0/2607263973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c14da80 con 0x7f7f3c154f70 2024-05-09T08:57:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f3b7fe640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7f3402ad00 tx=0x7f7f34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34009450 con 0x7f7f3c154f70 2024-05-09T08:57:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f34004240 con 0x7f7f3c154f70 2024-05-09T08:57:23.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340045f0 con 0x7f7f3c154f70 2024-05-09T08:57:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.684+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c0d0a80 con 0x7f7f3c154f70 2024-05-09T08:57:23.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.685+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c0d1000 con 0x7f7f3c154f70 2024-05-09T08:57:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.686+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c0b6800 con 0x7f7f3c154f70 2024-05-09T08:57:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.686+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f34039040 con 0x7f7f3c154f70 2024-05-09T08:57:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.687+0000 7f7f23fff640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 0x7f7f2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:23.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.687+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5857+0+0 (secure 0 0 0) 0x7f7f340ac250 con 0x7f7f3c154f70 2024-05-09T08:57:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.687+0000 7f7f3affd640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 0x7f7f2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:23.689 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.688+0000 7f7f3affd640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 0x7f7f2806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c005880 tx=0x7f7f2c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:23.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.691+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f3c0b6800 con 0x7f7f3c154f70 2024-05-09T08:57:23.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:23.738 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:23.739+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:23.812 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:23.812 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:23.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:23.828 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:23.830+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:23.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:23.896+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f7f3c04f660 con 0x7f7f3c154f70 2024-05-09T08:57:23.929 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:23.930+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:23.977 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.296739138000007, 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:23.978 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:23.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:23.982 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.296719167000006, 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:23.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.295890400999994, 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:23.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:23.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:23.991 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.294887509000006, 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:23.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:23.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:23.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.294584678000007, 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:23.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:23.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.25339812, 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:24.001 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:24.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.253386094000007, 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:24.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.253372623999994, 2024-05-09T08:57:24.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:24.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.015 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.252348221999995, 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:24.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.251920932999994, 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:24.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.251804593000003, 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:24.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:24.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.250530596999994, 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:24.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.248781197, 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:24.033 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:24.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.248453053000006, 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:24.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:24.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.042 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 72.238324425000002, 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:24.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:24.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.247581424000003, 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:24.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:24.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.247572038000001, 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:24.054 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:24.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.247560993999997, 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:24.060 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:24.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 66.245825998000001, 2024-05-09T08:57:24.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:24.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:24.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 54.215921792000003, 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:24.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:24.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:24.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:24.074 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:24.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:24.075 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:24.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:24.077 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:24.077 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:24.077+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:24.177 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:24.178+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:24.191 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:24.191 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:24.192 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:24.192 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:24.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:24.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.220+0000 7f5d4adb6640 1 Processor -- start 2024-05-09T08:57:24.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.220+0000 7f5d4adb6640 1 -- start start 2024-05-09T08:57:24.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d440cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d44154f70 0x7f5d440cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d440c6640 0x7f5d440c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5d440c6fd0 con 0x7f5d44154f70 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5d44072600 con 0x7f5d440c6640 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d4adb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440728e0 con 0x7f5d44154b90 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d44154f70 0x7f5d440cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d44154f70 0x7f5d440cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55260/0 (socket says 172.21.15.49:55260) 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 -- 172.21.15.49:0/116831131 learned_addr learned my addr 172.21.15.49:0/116831131 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d48b52640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d440cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d49353640 1 --2- 172.21.15.49:0/116831131 >> v2:172.21.15.183:3300/0 conn(0x7f5d440c6640 0x7f5d440c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 -- 172.21.15.49:0/116831131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 msgr2=0x7f5d440cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:24.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d440cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 -- 172.21.15.49:0/116831131 >> v2:172.21.15.183:3300/0 conn(0x7f5d440c6640 msgr2=0x7f5d440c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 --2- 172.21.15.49:0/116831131 >> v2:172.21.15.183:3300/0 conn(0x7f5d440c6640 0x7f5d440c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.221+0000 7f5d43fff640 1 -- 172.21.15.49:0/116831131 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440c5880 con 0x7f5d44154f70 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.222+0000 7f5d43fff640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44154f70 0x7f5d440cb7c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d440cdad0 tx=0x7f5d3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62de91a878e101de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.222+0000 7f5d437fe640 1 -- 172.21.15.49:0/116831131 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30037070 con 0x7f5d44154f70 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.222+0000 7f5d437fe640 1 -- 172.21.15.49:0/116831131 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d30032440 con 0x7f5d44154f70 2024-05-09T08:57:24.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.222+0000 7f5d437fe640 1 -- 172.21.15.49:0/116831131 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30032810 con 0x7f5d44154f70 2024-05-09T08:57:24.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 -- 172.21.15.49:0/116831131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44154f70 msgr2=0x7f5d440cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:24.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44154f70 0x7f5d440cb7c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d440cdad0 tx=0x7f5d3002a270 comp rx=0 tx=0).stop 2024-05-09T08:57:24.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 -- 172.21.15.49:0/116831131 shutdown_connections 2024-05-09T08:57:24.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/116831131 >> v2:172.21.15.183:3300/0 conn(0x7f5d440c6640 0x7f5d440c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d44154f70 0x7f5d440cb7c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/116831131 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d440cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 -- 172.21.15.49:0/116831131 >> 172.21.15.49:0/116831131 conn(0x7f5d44074840 msgr2=0x7f5d44074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 -- 172.21.15.49:0/116831131 shutdown_connections 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.223+0000 7f5d4adb6640 1 -- 172.21.15.49:0/116831131 wait complete. 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.224+0000 7f5d4adb6640 1 Processor -- start 2024-05-09T08:57:24.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.224+0000 7f5d4adb6640 1 -- start start 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.225+0000 7f5d4adb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d440c6640 0x7f5d441f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d4414baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44154f70 0x7f5d441e3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440c6120 con 0x7f5d440c6640 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440c5e40 con 0x7f5d44154f70 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5d4415af40 con 0x7f5d44154b90 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d440c6640 0x7f5d441f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d440c6640 0x7f5d441f0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55274/0 (socket says 172.21.15.49:55274) 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 -- 172.21.15.49:0/3872322392 learned_addr learned my addr 172.21.15.49:0/3872322392 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:24.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44154f70 0x7f5d441e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d4414baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 -- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 msgr2=0x7f5d4414baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d44154b90 0x7f5d4414baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 -- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44154f70 msgr2=0x7f5d441e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44154f70 0x7f5d441e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4414d660 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d440c6640 0x7f5d441f0d10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800e8a0 tx=0x7f5d3800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38017070 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3800c940 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800cd30 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.226+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4414c030 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.227+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4414c5c0 con 0x7f5d440c6640 2024-05-09T08:57:24.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.227+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5d4404f660 con 0x7f5d440c6640 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.228+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d3801a070 con 0x7f5d440c6640 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d417fa640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d2806be30 0x7f5d2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5857+0+0 (secure 0 0 0) 0x7f5d3808ad90 con 0x7f5d440c6640 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7f5d3808d410 con 0x7f5d440c6640 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d2806be30 0x7f5d2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5d441e0db0 0x7f5d441e1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.229+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d44158f50 con 0x7f5d441e0db0 2024-05-09T08:57:24.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.230+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5d441e0db0 0x7f5d441e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.230+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5d441e0db0 0x7f5d441e1190 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.230+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d2806be30 0x7f5d2806e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d30034000 tx=0x7f5d3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.231+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d44158f50 con 0x7f5d441e0db0 2024-05-09T08:57:24.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.247+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d44158c70 con 0x7f5d441e0db0 2024-05-09T08:57:24.247 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.248+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d44158c70 con 0x7f5d441e0db0 2024-05-09T08:57:24.248 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.249+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5d4414ce70 0x7f5d4414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.249+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5d4414d2b0 con 0x7f5d4414ce70 2024-05-09T08:57:24.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.249+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5d4414ce70 0x7f5d4414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.249+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5d4414ce70 0x7f5d4414a080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.250+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d4414d2b0 con 0x7f5d4414ce70 2024-05-09T08:57:24.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.265+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d4414da60 con 0x7f5d4414ce70 2024-05-09T08:57:24.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d4414da60 con 0x7f5d4414ce70 2024-05-09T08:57:24.266 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=43}) -- 0x7f5d4414a810 con 0x7f5d440c6640 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d4414dc70 con 0x7f5d440c6640 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5d38021040 con 0x7f5d440c6640 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d441f4b50 0x7f5d44061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.267+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f5d441f4f80 con 0x7f5d441f4b50 2024-05-09T08:57:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.268+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d441f4b50 0x7f5d44061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.268+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5d441f4b50 0x7f5d44061c70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.269+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d441f4f80 con 0x7f5d441f4b50 2024-05-09T08:57:24.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.284+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d44062480 con 0x7f5d441f4b50 2024-05-09T08:57:24.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.285+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d44062480 con 0x7f5d441f4b50 2024-05-09T08:57:24.284 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.285+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5d440b4b70 0x7f5d440b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.285+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f5d440b54f0 con 0x7f5d440b4b70 2024-05-09T08:57:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.286+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5d440b4b70 0x7f5d440b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.286+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5d440b4b70 0x7f5d440b4fb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.286 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:24.286+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:24.344 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:24.345 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:24.345 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:24.345 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:24.345 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:24.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.346+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d440b54f0 con 0x7f5d440b4b70 2024-05-09T08:57:24.353 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:24.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.362+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d4402c040 con 0x7f5d440b4b70 2024-05-09T08:57:24.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.364+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d4402c040 con 0x7f5d440b4b70 2024-05-09T08:57:24.363 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.364+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5d4402cb20 0x7f5d4402cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.364+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f5d4402c040 con 0x7f5d4402cb20 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.364+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5d4402cb20 0x7f5d4402cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.365+0000 7f5d48b52640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5d4402cb20 0x7f5d4402cf60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.365+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d4402c040 con 0x7f5d4402cb20 2024-05-09T08:57:24.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.380+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d4402c040 con 0x7f5d4402cb20 2024-05-09T08:57:24.380 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.381+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d4402c040 con 0x7f5d4402cb20 2024-05-09T08:57:24.381 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:24.381 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:24.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.382+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5d4402c040 0x7f5d44207b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.381 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.382+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f5d4420a8f0 con 0x7f5d4402c040 2024-05-09T08:57:24.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.382+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5d4402c040 0x7f5d44207b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.382+0000 7f5d43fff640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5d4402c040 0x7f5d44207b20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.383+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d4420a8f0 con 0x7f5d4402c040 2024-05-09T08:57:24.395 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:24.396+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:24.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.398+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d44208330 con 0x7f5d4402c040 2024-05-09T08:57:24.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.399+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d44208330 con 0x7f5d4402c040 2024-05-09T08:57:24.399 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:24.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.399+0000 7f5d4adb6640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5d44208a50 0x7f5d4420ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.399+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f5d4420dc10 con 0x7f5d44208a50 2024-05-09T08:57:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.400+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5d44208a50 0x7f5d4420ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.400+0000 7f5d49353640 1 --2- 172.21.15.49:0/3872322392 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5d44208a50 0x7f5d4420ae40 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:24.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.401+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d4420dc10 con 0x7f5d44208a50 2024-05-09T08:57:24.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.415+0000 7f5d4adb6640 1 -- 172.21.15.49:0/3872322392 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5d4420b380 con 0x7f5d44208a50 2024-05-09T08:57:24.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.417+0000 7f5d417fa640 1 -- 172.21.15.49:0/3872322392 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5d4420b380 con 0x7f5d44208a50 2024-05-09T08:57:24.416 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:24.417 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:24.420 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:24.420 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.473+0000 7f7f23fff640 1 -- 172.21.15.49:0/2607263973 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v43) ==== 91+0+0 (secure 0 0 0) 0x7f7f3c14da80 con 0x7f7f3c154f70 2024-05-09T08:57:24.473 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.475+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 msgr2=0x7f7f2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 0x7f7f2806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c005880 tx=0x7f7f2c009040 comp rx=0 tx=0).stop 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 msgr2=0x7f7f3c1f0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7f3402ad00 tx=0x7f7f34004a40 comp rx=0 tx=0).stop 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 shutdown_connections 2024-05-09T08:57:24.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f2806be30 0x7f7f2806e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f3c154f70 0x7f7f3c1f0d10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f3c154b90 0x7f7f3c14f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 --2- 172.21.15.49:0/2607263973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f3c0c6d50 0x7f7f3c14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 >> 172.21.15.49:0/2607263973 conn(0x7f7f3c074840 msgr2=0x7f7f3c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 shutdown_connections 2024-05-09T08:57:24.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:24.476+0000 7f7f41a38640 1 -- 172.21.15.49:0/2607263973 wait complete. 2024-05-09T08:57:24.480 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:24.480+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:24.500 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:24.500 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:24.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:24.581 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:24.582+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:24.654 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:24.655 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:24.666 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:24.686 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:24.687+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:24.783 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:24.784+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:24.791 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:24.791 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:24.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:24.884 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:24.885+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:24.908 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:24.908 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:24.917 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:24.985 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:24.986+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:25.024 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.024 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:25.089 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:25.089+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:25.138 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.138 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.146 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:25.195 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:25.196+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:25.253 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.253 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.261 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:25.288 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:25.289+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:25.389 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:25.390+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:25.410 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.410 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.419 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:25.490 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:25.491+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:25.565 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.565 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.574 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:25.617 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:25.618+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:25.675 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.676 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.684 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:25.726 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:25.727+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:25.793 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:25.794+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:25.802 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.802 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.812 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:25.894 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:25.896+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:25.922 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:25.923 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:25.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:25.994 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:25.996+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:26.044 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:26.045 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:26.054 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:26.095 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:26.097+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:26.174 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:26.174 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:26.183 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:26.196 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:26.197+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:26.254 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:26.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:26.255 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-05-09T08:57:26.255 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-09T08:57:26.255 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-09T08:57:26.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-05-09T08:57:26.296 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:26.297+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:26.297 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:26.298 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:26.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:26.361 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:26.362 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-09T08:57:26.362 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:26.398 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:26.399+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:26.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:26.466 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:26.467 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:26.478 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:26.492 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:57:26.522 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:26.523+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:26.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.537+0000 7f2cc5e9a640 1 Processor -- start 2024-05-09T08:57:26.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.537+0000 7f2cc5e9a640 1 -- start start 2024-05-09T08:57:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cc5e9a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cc00cd280 0x7f2cc00c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cc5e9a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cc5e9a640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cc00cdc50 con 0x7f2cc0151c00 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cc5e9a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2cc0157ac0 con 0x7f2cc00cd280 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cc5e9a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc0157da0 con 0x7f2cc00cd710 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cbeffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cc00cd280 0x7f2cc00c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cbeffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2cc00cd280 0x7f2cc00c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55278/0 (socket says 172.21.15.49:55278) 2024-05-09T08:57:26.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.538+0000 7f2cbeffd640 1 -- 172.21.15.49:0/1672810565 learned_addr learned my addr 172.21.15.49:0/1672810565 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbeffd640 1 -- 172.21.15.49:0/1672810565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 msgr2=0x7f2cc00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbffff640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc00c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbeffd640 1 -- 172.21.15.49:0/1672810565 >> v1:172.21.15.183:6789/0 conn(0x7f2cc0151c00 legacy=0x7f2cc00c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbeffd640 1 -- 172.21.15.49:0/1672810565 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc0158080 con 0x7f2cc00cd280 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbffff640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.539+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc00cd280 0x7f2cc00c8180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400bb10 tx=0x7f2cb402c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cefbcd90b2599a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.540+0000 7f2cbe7fc640 1 -- 172.21.15.49:0/1672810565 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4036020 con 0x7f2cc00cd280 2024-05-09T08:57:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.540+0000 7f2cbe7fc640 1 -- 172.21.15.49:0/1672810565 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb4032820 con 0x7f2cc00cd280 2024-05-09T08:57:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.540+0000 7f2cbe7fc640 1 -- 172.21.15.49:0/1672810565 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4032bf0 con 0x7f2cc00cd280 2024-05-09T08:57:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.540+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/1672810565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc00cd280 msgr2=0x7f2cc00c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.540+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc00cd280 0x7f2cc00c8180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2cb400bb10 tx=0x7f2cb402c960 comp rx=0 tx=0).stop 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/1672810565 shutdown_connections 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/1672810565 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc00cd280 0x7f2cc00c8180 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/1672810565 >> 172.21.15.49:0/1672810565 conn(0x7f2cc006b200 msgr2=0x7f2cc006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/1672810565 shutdown_connections 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.541+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/1672810565 wait complete. 2024-05-09T08:57:26.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.542+0000 7f2cc5e9a640 1 Processor -- start 2024-05-09T08:57:26.542 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.542+0000 7f2cc5e9a640 1 -- start start 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cc00cd280 0x7f2cc0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc01f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc0151c00 0x7f2cc00c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0151fe0 con 0x7f2cc0151c00 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0158640 con 0x7f2cc00cd280 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cc5e9a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc015b0a0 con 0x7f2cc00cd710 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc0151c00 0x7f2cc00c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc0151c00 0x7f2cc00c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55286/0 (socket says 172.21.15.49:55286) 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cbffff640 1 -- 172.21.15.49:0/3540864065 learned_addr learned my addr 172.21.15.49:0/3540864065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc01f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.543+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cc00cd280 0x7f2cc0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 -- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 msgr2=0x7f2cc01f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc01f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 -- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cc00cd280 msgr2=0x7f2cc0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2cc00cd280 0x7f2cc0159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01571f0 con 0x7f2cc0151c00 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2cc00cd710 0x7f2cc01f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:26.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2cc0151c00 0x7f2cc00c5c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800cef0 tx=0x7f2cb800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb801a070 con 0x7f2cc0151c00 2024-05-09T08:57:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cb800d860 con 0x7f2cc0151c00 2024-05-09T08:57:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01574d0 con 0x7f2cc0151c00 2024-05-09T08:57:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800dc50 con 0x7f2cc0151c00 2024-05-09T08:57:26.545 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.544+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc0159ac0 con 0x7f2cc0151c00 2024-05-09T08:57:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.546+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2cc004f660 con 0x7f2cc0151c00 2024-05-09T08:57:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.546+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cb801d070 con 0x7f2cc0151c00 2024-05-09T08:57:26.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.547+0000 7f2ca7fff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2cac06be30 0x7f2cac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.547+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5857+0+0 (secure 0 0 0) 0x7f2cb808b630 con 0x7f2cc0151c00 2024-05-09T08:57:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.547+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2cac06be30 0x7f2cac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.547+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7f2cb8091170 con 0x7f2cc0151c00 2024-05-09T08:57:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.548+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f2cc014fb90 0x7f2cc014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.548+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cc00c8760 con 0x7f2cc014fb90 2024-05-09T08:57:26.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.548+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f2cc014fb90 0x7f2cc014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.548+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f2cc014fb90 0x7f2cc014cf20 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.548+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2cac06be30 0x7f2cac06e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0006fd0 tx=0x7f2cb0009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.548 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.549+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc00c8760 con 0x7f2cc014fb90 2024-05-09T08:57:26.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.577+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc00c8d10 con 0x7f2cc014fb90 2024-05-09T08:57:26.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.580+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc00c8d10 con 0x7f2cc014fb90 2024-05-09T08:57:26.580 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:26.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:26.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.581+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f2cc014d970 0x7f2cc00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.581+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2cc0150680 con 0x7f2cc014d970 2024-05-09T08:57:26.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.581+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f2cc014d970 0x7f2cc00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.581+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f2cc014d970 0x7f2cc00cc160 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.582+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc0150680 con 0x7f2cc014d970 2024-05-09T08:57:26.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.600+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc00caad0 con 0x7f2cc014d970 2024-05-09T08:57:26.600 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:26.601+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:26.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.603+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc00caad0 con 0x7f2cc014d970 2024-05-09T08:57:26.602 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:26.603 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.603+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=44}) -- 0x7f2cc01576a0 con 0x7f2cc0151c00 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.603+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2cc014f6e0 con 0x7f2cc0151c00 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.604+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2cb8092070 con 0x7f2cc0151c00 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.604+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f2cc00ccbb0 0x7f2cc0061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.604+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f2cc00ccfc0 con 0x7f2cc00ccbb0 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.604+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f2cc00ccbb0 0x7f2cc0061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.604+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f2cc00ccbb0 0x7f2cc0061c70 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.605+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc00ccfc0 con 0x7f2cc00ccbb0 2024-05-09T08:57:26.633 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:26.633 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:26.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.639+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc00621b0 con 0x7f2cc00ccbb0 2024-05-09T08:57:26.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.640+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc00621b0 con 0x7f2cc00ccbb0 2024-05-09T08:57:26.640 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:26.640 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:26.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.641+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f2cc0155740 0x7f2cc0155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.641+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f2cc01560a0 con 0x7f2cc0155740 2024-05-09T08:57:26.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.641+0000 7f5cca838640 1 Processor -- start 2024-05-09T08:57:26.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.641+0000 7f5cca838640 1 -- start start 2024-05-09T08:57:26.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.641+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f2cc0155740 0x7f2cc0155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f2cc0155740 0x7f2cc0155b60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cca838640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5cc4154f70 0x7f5cc40c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cca838640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc40c6b80 0x7f5cc4071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cca838640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc4072600 con 0x7f5cc4154b90 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cca838640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc40728e0 con 0x7f5cc4154f70 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cca838640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc40c5880 con 0x7f5cc40c6b80 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cc37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5cc4154f70 0x7f5cc40c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cc37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5cc4154f70 0x7f5cc40c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55298/0 (socket says 172.21.15.49:55298) 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cc37fe640 1 -- 172.21.15.49:0/2262508811 learned_addr learned my addr 172.21.15.49:0/2262508811 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f5cc8dd5640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc40c6b80 0x7f5cc4071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.642+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc01560a0 con 0x7f2cc0155740 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.643+0000 7f5cc37fe640 1 -- 172.21.15.49:0/2262508811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc40c6b80 msgr2=0x7f5cc4071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.643+0000 7f5cc37fe640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc40c6b80 0x7f5cc4071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.643+0000 7f5cc37fe640 1 -- 172.21.15.49:0/2262508811 >> v1:172.21.15.183:6789/0 conn(0x7f5cc4154b90 legacy=0x7f5cc40cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.643+0000 7f5cc37fe640 1 -- 172.21.15.49:0/2262508811 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc40c5b60 con 0x7f5cc4154f70 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.644+0000 7f5cc37fe640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154f70 0x7f5cc40c6640 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009870 tx=0x7f5cb802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec83e0cbda42fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.644+0000 7f5cc2ffd640 1 -- 172.21.15.49:0/2262508811 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8037070 con 0x7f5cc4154f70 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.644+0000 7f5cc2ffd640 1 -- 172.21.15.49:0/2262508811 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb8032440 con 0x7f5cc4154f70 2024-05-09T08:57:26.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.644+0000 7f5cc2ffd640 1 -- 172.21.15.49:0/2262508811 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8032810 con 0x7f5cc4154f70 2024-05-09T08:57:26.646 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:26.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.645+0000 7f5cca838640 1 -- 172.21.15.49:0/2262508811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154f70 msgr2=0x7f5cc40c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.645+0000 7f5cca838640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154f70 0x7f5cc40c6640 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009870 tx=0x7f5cb802a200 comp rx=0 tx=0).stop 2024-05-09T08:57:26.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 -- 172.21.15.49:0/2262508811 shutdown_connections 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc40c6b80 0x7f5cc4071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 --2- 172.21.15.49:0/2262508811 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154f70 0x7f5cc40c6640 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 -- 172.21.15.49:0/2262508811 >> 172.21.15.49:0/2262508811 conn(0x7f5cc4074840 msgr2=0x7f5cc4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 -- 172.21.15.49:0/2262508811 shutdown_connections 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.646+0000 7f5cca838640 1 -- 172.21.15.49:0/2262508811 wait complete. 2024-05-09T08:57:26.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.648+0000 7f5cca838640 1 Processor -- start 2024-05-09T08:57:26.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.648+0000 7f5cca838640 1 -- start start 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc40c6b80 0x7f5cc4145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc4154f70 0x7f5cc41f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc40c6fa0 con 0x7f5cc4154b90 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc40c6120 con 0x7f5cc40c6b80 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cca838640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc415af40 con 0x7f5cc4154f70 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cc37fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55304/0 (socket says 172.21.15.49:55304) 2024-05-09T08:57:26.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cc37fe640 1 -- 172.21.15.49:0/3277665707 learned_addr learned my addr 172.21.15.49:0/3277665707 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cc8dd5640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc4154f70 0x7f5cc41f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.649+0000 7f5cc3fff640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc40c6b80 0x7f5cc4145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 -- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc4154f70 msgr2=0x7f5cc41f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc4154f70 0x7f5cc41f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 -- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc40c6b80 msgr2=0x7f5cc4145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc40c6b80 0x7f5cc4145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 -- 172.21.15.49:0/3277665707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41454e0 con 0x7f5cc4154b90 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc37fe640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8042070 con 0x7f5cc4154b90 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb8004230 con 0x7f5cc4154b90 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8037040 con 0x7f5cc4154b90 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc40cf720 con 0x7f5cc4154b90 2024-05-09T08:57:26.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.650+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc40cfce0 con 0x7f5cc4154b90 2024-05-09T08:57:26.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.652+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc404f660 con 0x7f5cc4154b90 2024-05-09T08:57:26.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.653+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cb8045070 con 0x7f5cc4154b90 2024-05-09T08:57:26.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.653+0000 7f5cc0ff9640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 0x7f5cb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.653+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5857+0+0 (secure 0 0 0) 0x7f5cb80aac50 con 0x7f5cc4154b90 2024-05-09T08:57:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.654+0000 7f5cc3fff640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 0x7f5cb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.654+0000 7f5cc3fff640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 0x7f5cb006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000f60 tx=0x7f5cb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.659+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5cb80b0020 con 0x7f5cc4154b90 2024-05-09T08:57:26.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.663+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc01565b0 con 0x7f2cc0155740 2024-05-09T08:57:26.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.665+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc01565b0 con 0x7f2cc0155740 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.665+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f2cc002c550 0x7f2cc002c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.665+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f2cc002ced0 con 0x7f2cc002c550 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.666+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f2cc002c550 0x7f2cc002c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.665 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.666+0000 7f2cbf7fe640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f2cc002c550 0x7f2cc002c990 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.666 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.667+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc002ced0 con 0x7f2cc002c550 2024-05-09T08:57:26.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.685+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc002ced0 con 0x7f2cc002c550 2024-05-09T08:57:26.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.687+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc002ced0 con 0x7f2cc002c550 2024-05-09T08:57:26.686 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:26.686 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:26.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.687+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f2cc02055a0 0x7f2cc0207990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.687+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f2cc002d3e0 con 0x7f2cc02055a0 2024-05-09T08:57:26.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.687+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f2cc02055a0 0x7f2cc0207990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.687+0000 7f2cbeffd640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f2cc02055a0 0x7f2cc0207990 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.688+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc002d3e0 con 0x7f2cc02055a0 2024-05-09T08:57:26.700 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:26.701+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:26.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.704+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc002d3e0 con 0x7f2cc02055a0 2024-05-09T08:57:26.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.706+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc002d3e0 con 0x7f2cc02055a0 2024-05-09T08:57:26.705 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.706+0000 7f2cc5e9a640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f2cc02083e0 0x7f2cc020a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.706+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f2cc020d650 con 0x7f2cc02083e0 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.707+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f2cc02083e0 0x7f2cc020a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.707+0000 7f2cbffff640 1 --2- 172.21.15.49:0/3540864065 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f2cc02083e0 0x7f2cc020a7d0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:26.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.708+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2cc020d650 con 0x7f2cc02083e0 2024-05-09T08:57:26.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.724+0000 7f2cc5e9a640 1 -- 172.21.15.49:0/3540864065 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2cc020ad90 con 0x7f2cc02083e0 2024-05-09T08:57:26.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.725+0000 7f2ca7fff640 1 -- 172.21.15.49:0/3540864065 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2cc020ad90 con 0x7f2cc02083e0 2024-05-09T08:57:26.725 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:26.725 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:26.728 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:26.764 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:26.764 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:26.764 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:26.764 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:26.764 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:26.774 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:26.816 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:26.817+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:26.903 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:26.905+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:26.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:26.913+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f5cc40cbde0 con 0x7f5cc4154b90 2024-05-09T08:57:26.931 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:26.931 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.250507755000001, 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:26.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.250487784000001, 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:26.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:26.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:26.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.249659018000003, 2024-05-09T08:57:26.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:26.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:26.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.947 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.248656126, 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.948 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:26.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:26.953 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.248353295000001, 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:26.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.207166736999994, 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:26.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:26.966 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.207154711000001, 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:26.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.972 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.207141241000002, 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:26.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:26.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.206116839000003, 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:26.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:26.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:26.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:26.982 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.205689550000002, 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:26.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:26.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.987 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.205573209999997, 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:26.988 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.204299214000002, 2024-05-09T08:57:26.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:26.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:26.996 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.202549813999994, 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:26.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.20222167, 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:27.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:27.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:27.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.006 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.192093041999996, 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:27.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:27.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.201350040999998, 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:27.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.201340654999996, 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:27.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:27.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:27.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:27.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:27.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.201329611000006, 2024-05-09T08:57:27.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.199594614999995, 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.032 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:27.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.037 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.169690408999998, 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:27.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:27.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:27.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:27.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:27.043 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:27.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:27.046 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:27.046+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:27.146 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:27.147+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:27.151 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.151 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:27.152 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:27.152 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:27.160 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:27.247 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:27.248+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:27.261 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.261 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:27.261 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:27.261 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:27.262 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:27.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:27.347 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:27.348+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:27.373 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.693593996999994, 2024-05-09T08:57:27.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.690174235000001, 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.382 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.689947726, 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:27.388 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:27.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:27.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:27.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.689894433000006, 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:27.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:27.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:27.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:27.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:27.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.641570861999995, 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:27.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:27.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:27.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.641044987000001, 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.406 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:27.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.640797655, 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:27.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:27.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.416 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.635358452000006, 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:27.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:27.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.627399287999999, 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.626835108999998, 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:27.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:27.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.433 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.626748036000002, 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.434 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:27.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.626678978000001, 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.439 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:27.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:27.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.625320668999997, 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:27.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:27.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.625275422000001, 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:27.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:27.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:27.455 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.625020155000001, 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:27.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 63.624981458999997, 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:27.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.612038401, 2024-05-09T08:57:27.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:27.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.611399452999997, 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:27.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:27.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:27.477 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.610845509000001, 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.607991370999997, 2024-05-09T08:57:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.489 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:27.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.490+0000 7f5cc0ff9640 1 -- 172.21.15.49:0/3277665707 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v44) ==== 101+0+0 (secure 0 0 0) 0x7f5cb8078e80 con 0x7f5cc4154b90 2024-05-09T08:57:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:57:27.491 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:27.491+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:27.492 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 msgr2=0x7f5cb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 0x7f5cb006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4000f60 tx=0x7f5cb4009040 comp rx=0 tx=0).stop 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 msgr2=0x7f5cc4146200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000c00 tx=0x7f5cb80046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 shutdown_connections 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cb006be30 0x7f5cb006e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5cc4154f70 0x7f5cc41f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:27.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5cc4154b90 0x7f5cc4146200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 --2- 172.21.15.49:0/3277665707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc40c6b80 0x7f5cc4145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.494+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 >> 172.21.15.49:0/3277665707 conn(0x7f5cc4074840 msgr2=0x7f5cc4149ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.495+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 shutdown_connections 2024-05-09T08:57:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:27.495+0000 7f5cca838640 1 -- 172.21.15.49:0/3277665707 wait complete. 2024-05-09T08:57:27.592 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:27.593+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:27.636 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.636 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:27.636 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:27.636 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:27.645 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:27.729 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:27.730+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:27.791 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.791 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:27.792 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:27.792 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:27.792 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:27.793 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:27.794+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:27.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:27.894 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:27.895+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:27.941 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:27.941 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.216425018999999, 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.942 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:27.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.216404420000003, 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.949 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:27.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:27.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.216399549000002, 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:27.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:27.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:27.960 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.212841249999997, 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:27.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.212833021999998, 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:27.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:27.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.973 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.212829213000006, 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:27.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:27.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.210273185000005, 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:27.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:27.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.988 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.195772378000001, 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:27.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:27.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:27.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:27.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.195756907000003, 2024-05-09T08:57:27.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:27.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:27.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:28.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.195752162999995, 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.003 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:28.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.195745746, 2024-05-09T08:57:28.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:28.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:28.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.015 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179769526999998, 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:28.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179756824999998, 2024-05-09T08:57:28.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:28.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179748908000001, 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.027 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:28.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:28.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179746399999999, 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:28.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:28.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179743936000001, 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:28.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:28.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.044 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179728640999997, 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:28.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179724319999998, 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:28.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.179721180000001, 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:28.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.177261379999997, 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:28.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.063 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:28.065 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:28.065+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:28.166 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:28.167+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:28.208 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:28.209 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:28.209 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:28.267 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:28.268+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:28.320 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:28.320 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:28.320 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:28.320 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:28.321 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.329 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:28.369 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:28.370+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:28.468 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:28.469+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:28.475 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.794503445999993, 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:28.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:28.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.794500579000001, 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:28.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:28.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:28.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 81.794496933999994, 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.751687769, 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.495 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:28.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.746895780000003, 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:28.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:28.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:28.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.744389412000004, 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:28.510 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:28.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:28.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.742918021999998, 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:28.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.519 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 70.742842183999997, 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.520 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:28.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.730020773999996, 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.525 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:28.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:28.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:28.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:28.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:28.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729999776, 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:28.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:28.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:28.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.543 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729995381999998, 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:28.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729990323999999, 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:28.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729986843000006, 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:28.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:28.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.562 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729982225000001, 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:28.563 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:28.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:28.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:28.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729978755999994, 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:28.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729975455000002, 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:28.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:28.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:28.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 64.729662386000001, 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:28.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:28.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.713586564000003, 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.588 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:28.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.713581884999996, 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:28.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 58.713575867000003, 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:28.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:28.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:28.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:28.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:28.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:28.609 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.611 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:28.611 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:28.612+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:28.711 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:28.712+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:28.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:28.755 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:28.755 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:28.755 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:28.756 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:28.812 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:28.812+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:28.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.836+0000 7f63793f9640 1 Processor -- start 2024-05-09T08:57:28.837 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.837+0000 7f63793f9640 1 -- start start 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.837+0000 7f63793f9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 0x7f63740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.837+0000 7f63793f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63740cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63793f9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6374072600 con 0x7f6374154f70 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63793f9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f63740728e0 con 0x7f6374154b90 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63793f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f63740c5880 con 0x7f63740c6d50 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63727fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f6374154f70 0x7f63740c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57728/0 (socket says 172.21.15.49:57728) 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63727fc640 1 -- 172.21.15.49:0/4056146377 learned_addr learned my addr 172.21.15.49:0/4056146377 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f6372ffd640 1 --2- 172.21.15.49:0/4056146377 >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 0x7f63740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63737fe640 1 --2- 172.21.15.49:0/4056146377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63740cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f6371ffb640 1 -- 172.21.15.49:0/4056146377 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465886613 0 0) 0x7f6374072600 con 0x7f6374154f70 2024-05-09T08:57:28.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f6371ffb640 1 -- 172.21.15.49:0/4056146377 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f635c003520 con 0x7f6374154f70 2024-05-09T08:57:28.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63737fe640 1 -- 172.21.15.49:0/4056146377 >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 msgr2=0x7f63740cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:28.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63737fe640 1 --2- 172.21.15.49:0/4056146377 >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 0x7f63740cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63737fe640 1 -- 172.21.15.49:0/4056146377 >> v1:172.21.15.49:6789/0 conn(0x7f6374154f70 legacy=0x7f63740c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:28.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f63737fe640 1 -- 172.21.15.49:0/4056146377 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63740c5b60 con 0x7f63740c6d50 2024-05-09T08:57:28.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.838+0000 7f6372ffd640 1 --2- 172.21.15.49:0/4056146377 >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 0x7f63740cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:28.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.839+0000 7f63737fe640 1 --2- 172.21.15.49:0/4056146377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63740cb7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f636c009d90 tx=0x7f636c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98f194e3d53fc41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.839+0000 7f6371ffb640 1 -- 172.21.15.49:0/4056146377 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c007740 con 0x7f63740c6d50 2024-05-09T08:57:28.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.839+0000 7f6371ffb640 1 -- 172.21.15.49:0/4056146377 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636c0078e0 con 0x7f63740c6d50 2024-05-09T08:57:28.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.839+0000 7f6371ffb640 1 -- 172.21.15.49:0/4056146377 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c007cb0 con 0x7f63740c6d50 2024-05-09T08:57:28.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 -- 172.21.15.49:0/4056146377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 msgr2=0x7f63740cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 --2- 172.21.15.49:0/4056146377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63740cb7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f636c009d90 tx=0x7f636c02a6f0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 -- 172.21.15.49:0/4056146377 shutdown_connections 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 --2- 172.21.15.49:0/4056146377 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63740cb7c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 --2- 172.21.15.49:0/4056146377 >> v2:172.21.15.183:3300/0 conn(0x7f6374154b90 0x7f63740cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 -- 172.21.15.49:0/4056146377 >> 172.21.15.49:0/4056146377 conn(0x7f6374074840 msgr2=0x7f6374074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 -- 172.21.15.49:0/4056146377 shutdown_connections 2024-05-09T08:57:28.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.840+0000 7f63793f9640 1 -- 172.21.15.49:0/4056146377 wait complete. 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.841+0000 7f63793f9640 1 Processor -- start 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 -- start start 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63741f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 0x7f63741524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374154f70 0x7f63741f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f637415af40 con 0x7f6374154b90 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63740c6120 con 0x7f6374154f70 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63793f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6374071820 con 0x7f63740c6d50 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63727fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 0x7f63741524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63727fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 0x7f63741524e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55316/0 (socket says 172.21.15.49:55316) 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f63727fc640 1 -- 172.21.15.49:0/2863617175 learned_addr learned my addr 172.21.15.49:0/2863617175 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.842+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63741f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374154f70 0x7f63741f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 -- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374154f70 msgr2=0x7f63741f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374154f70 0x7f63741f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 -- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 msgr2=0x7f63741524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 0x7f63741524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6374159b90 con 0x7f63740c6d50 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6374154b90 0x7f63741524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6374154f70 0x7f63741f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:28.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63740c6d50 0x7f63741f41a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f636400e900 tx=0x7f636400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364017070 con 0x7f63740c6d50 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636400ca00 con 0x7f63740c6d50 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6374159e50 con 0x7f63740c6d50 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636400cdf0 con 0x7f63740c6d50 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.843+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741f18e0 con 0x7f63740c6d50 2024-05-09T08:57:28.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.845+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f637404f660 con 0x7f63740c6d50 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.845+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f636401a070 con 0x7f63740c6d50 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f635b7fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5857+0+0 (secure 0 0 0) 0x7f636408b4a0 con 0x7f63740c6d50 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7f636408da80 con 0x7f63740c6d50 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6374153310 0x7f637414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.846+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63740cbde0 con 0x7f6374153310 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.847+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f636006be30 0x7f636006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63740cab60 tx=0x7f6368009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.847+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6374153310 0x7f637414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.847+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f6374153310 0x7f637414a3c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.848+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63740cbde0 con 0x7f6374153310 2024-05-09T08:57:28.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.863+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f63740cc250 con 0x7f6374153310 2024-05-09T08:57:28.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.865+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f63740cc250 con 0x7f6374153310 2024-05-09T08:57:28.865 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.865+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f63741536f0 0x7f637414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.865+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6374159850 con 0x7f63741536f0 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.865+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f63741536f0 0x7f637414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.865+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f63741536f0 0x7f637414c100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.866+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6374159850 con 0x7f63741536f0 2024-05-09T08:57:28.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.885+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6374152a20 con 0x7f63741536f0 2024-05-09T08:57:28.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.886+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6374152a20 con 0x7f63741536f0 2024-05-09T08:57:28.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=45}) -- 0x7f637414c640 con 0x7f63740c6d50 2024-05-09T08:57:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f637414c9e0 con 0x7f63740c6d50 2024-05-09T08:57:28.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6364021040 con 0x7f63740c6d50 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6374062180 0x7f63740625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f6374062ae0 con 0x7f6374062180 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.887+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6374062180 0x7f63740625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.888+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f6374062180 0x7f63740625a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.889+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6374062ae0 con 0x7f6374062180 2024-05-09T08:57:28.907 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:28.907 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:28.907 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:28.912 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:28.912+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:28.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:28.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.918+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f63740b4880 con 0x7f6374062180 2024-05-09T08:57:28.919 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.919+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f63740b4880 con 0x7f6374062180 2024-05-09T08:57:28.920 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.919+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f63740b5360 0x7f637402c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.919+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f6374204210 con 0x7f63740b5360 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.920+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f63740b5360 0x7f637402c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.920+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f63740b5360 0x7f637402c040 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.921+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6374204210 con 0x7f63740b5360 2024-05-09T08:57:28.935 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.935+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f637402c850 con 0x7f63740b5360 2024-05-09T08:57:28.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.937+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f637402c850 con 0x7f63740b5360 2024-05-09T08:57:28.937 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:28.938 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:28.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.937+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f637402cf70 0x7f6374204b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.937+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f637402c7d0 con 0x7f637402cf70 2024-05-09T08:57:28.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.937+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f637402cf70 0x7f6374204b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.938+0000 7f6372ffd640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f637402cf70 0x7f6374204b10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.938+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f637402c7d0 con 0x7f637402cf70 2024-05-09T08:57:28.953 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.953+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f637402c580 con 0x7f637402cf70 2024-05-09T08:57:28.954 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.954+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f637402c580 con 0x7f637402cf70 2024-05-09T08:57:28.955 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.955+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f637402c580 0x7f6374207570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.955+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f637420a340 con 0x7f637402c580 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.955+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f637402c580 0x7f6374207570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.955+0000 7f63727fc640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f637402c580 0x7f6374207570 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.956 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.956+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f637420a340 con 0x7f637402c580 2024-05-09T08:57:28.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.970+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6374207ab0 con 0x7f637402c580 2024-05-09T08:57:28.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.972+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6374207ab0 con 0x7f637402c580 2024-05-09T08:57:28.972 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:28.973 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.972+0000 7f63793f9640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6374208130 0x7f637420a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.972+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f637420d3f0 con 0x7f6374208130 2024-05-09T08:57:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.972+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6374208130 0x7f637420a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.973+0000 7f63737fe640 1 --2- 172.21.15.49:0/2863617175 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f6374208130 0x7f637420a520 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:28.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:28.973+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f637420d3f0 con 0x7f6374208130 2024-05-09T08:57:29.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.003+0000 7f63793f9640 1 -- 172.21.15.49:0/2863617175 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f637420aa60 con 0x7f6374208130 2024-05-09T08:57:29.004 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.004+0000 7f635b7fe640 1 -- 172.21.15.49:0/2863617175 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f637420aa60 con 0x7f6374208130 2024-05-09T08:57:29.005 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:29.005 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:29.009 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:29.009 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:29.012 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:29.014+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:29.020 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.020 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:29.113 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:29.115+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:29.160 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.161 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:29.214 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:29.214+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:29.316 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:29.316+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:29.324 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.324 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:29.420 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:29.420+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:29.497 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.497 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.507 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:57:29.509 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:29.552 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:29.552+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:29.620 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:29.621+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.637+0000 7f75a4f34640 1 Processor -- start 2024-05-09T08:57:29.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.637+0000 7f75a4f34640 1 -- start start 2024-05-09T08:57:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75a0154f70 0x7f75a00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f75a00c6640 0x7f75a00c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f75a00c6fd0 con 0x7f75a0154f70 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f75a0072600 con 0x7f75a00c6640 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.638+0000 7f75a4f34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75a00728e0 con 0x7f75a0154b90 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75a0154f70 0x7f75a00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f75a0154f70 0x7f75a00cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55318/0 (socket says 172.21.15.49:55318) 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 -- 172.21.15.49:0/3431812810 learned_addr learned my addr 172.21.15.49:0/3431812810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759e59c640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759ed9d640 1 --2- 172.21.15.49:0/3431812810 >> v2:172.21.15.183:3300/0 conn(0x7f75a00c6640 0x7f75a00c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 -- 172.21.15.49:0/3431812810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 msgr2=0x7f75a00cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 -- 172.21.15.49:0/3431812810 >> v2:172.21.15.183:3300/0 conn(0x7f75a00c6640 msgr2=0x7f75a00c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:29.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 --2- 172.21.15.49:0/3431812810 >> v2:172.21.15.183:3300/0 conn(0x7f75a00c6640 0x7f75a00c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759dd9b640 1 -- 172.21.15.49:0/3431812810 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a00c5880 con 0x7f75a0154f70 2024-05-09T08:57:29.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.639+0000 7f759e59c640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.640+0000 7f759dd9b640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00cb7c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cdad0 tx=0x7f759402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d8e47e0fa13aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.641+0000 7f759d59a640 1 -- 172.21.15.49:0/3431812810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594037070 con 0x7f75a0154f70 2024-05-09T08:57:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.641+0000 7f759d59a640 1 -- 172.21.15.49:0/3431812810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7594032440 con 0x7f75a0154f70 2024-05-09T08:57:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.641+0000 7f759d59a640 1 -- 172.21.15.49:0/3431812810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7594032810 con 0x7f75a0154f70 2024-05-09T08:57:29.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.641+0000 7f75a4f34640 1 -- 172.21.15.49:0/3431812810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 msgr2=0x7f75a00cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.641+0000 7f75a4f34640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00cb7c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75a00cdad0 tx=0x7f759402a270 comp rx=0 tx=0).stop 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 -- 172.21.15.49:0/3431812810 shutdown_connections 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 --2- 172.21.15.49:0/3431812810 >> v2:172.21.15.183:3300/0 conn(0x7f75a00c6640 0x7f75a00c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00cb7c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 --2- 172.21.15.49:0/3431812810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 -- 172.21.15.49:0/3431812810 >> 172.21.15.49:0/3431812810 conn(0x7f75a0074840 msgr2=0x7f75a0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 -- 172.21.15.49:0/3431812810 shutdown_connections 2024-05-09T08:57:29.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.642+0000 7f75a4f34640 1 -- 172.21.15.49:0/3431812810 wait complete. 2024-05-09T08:57:29.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.644+0000 7f75a4f34640 1 Processor -- start 2024-05-09T08:57:29.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.644+0000 7f75a4f34640 1 -- start start 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.644+0000 7f75a4f34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75a00c6640 0x7f75a0155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.644+0000 7f75a4f34640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.644+0000 7f75a4f34640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f75a4f34640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f75a00c6120 con 0x7f75a0154f70 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f75a4f34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75a00c5e40 con 0x7f75a00c6640 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f75a4f34640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f75a015af40 con 0x7f75a0154b90 2024-05-09T08:57:29.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75a00c6640 0x7f75a0155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55328/0 (socket says 172.21.15.49:55328) 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 -- 172.21.15.49:0/1602616331 learned_addr learned my addr 172.21.15.49:0/1602616331 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759dd9b640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 -- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 msgr2=0x7f75a00c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 -- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75a00c6640 msgr2=0x7f75a0155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75a00c6640 0x7f75a0155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 -- 172.21.15.49:0/1602616331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a00c92d0 con 0x7f75a0154f70 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.645+0000 7f759ed9d640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f759800efc0 tx=0x7f759800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.646+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598018020 con 0x7f75a0154f70 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.646+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7598014480 con 0x7f75a0154f70 2024-05-09T08:57:29.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.646+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598014870 con 0x7f75a0154f70 2024-05-09T08:57:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.646+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a0155a90 con 0x7f75a0154f70 2024-05-09T08:57:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.646+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a0158170 con 0x7f75a0154f70 2024-05-09T08:57:29.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.647+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a00c94a0 con 0x7f75a0154f70 2024-05-09T08:57:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.647+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7598005d10 con 0x7f75a0154f70 2024-05-09T08:57:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.648+0000 7f758effd640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.648+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5857+0+0 (secure 0 0 0) 0x7f759808b8b0 con 0x7f75a0154f70 2024-05-09T08:57:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.648+0000 7f759e59c640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:29.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.649+0000 7f759e59c640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 0x7f758006e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7590006e80 tx=0x7f7590008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:29.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.656+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7598022070 con 0x7f75a0154f70 2024-05-09T08:57:29.682 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.682 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.696 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:29.737 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:29.737+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:29.821 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:29.821+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:29.856 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:29.856 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:29.865 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:29.950 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:29.951+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:29.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:29.984+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f75a004f660 con 0x7f75a0154f70 2024-05-09T08:57:30.023 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:30.024+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:30.061 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.061 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:30.154 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:30.155+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:30.224 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:30.226+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:30.266 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.266 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.278 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:30.326 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:30.327+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:30.427 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:30.428+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:30.429 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.429 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:30.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.509+0000 7f758effd640 1 -- 172.21.15.49:0/1602616331 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v45) ==== 87+0+0 (secure 0 0 0) 0x7f7598010040 con 0x7f75a0154f70 2024-05-09T08:57:30.510 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:57:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 msgr2=0x7f758006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 0x7f758006e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7590006e80 tx=0x7f7590008040 comp rx=0 tx=0).stop 2024-05-09T08:57:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 msgr2=0x7f75a00c97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:30.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f759800efc0 tx=0x7f759800c490 comp rx=0 tx=0).stop 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 shutdown_connections 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f758006be30 0x7f758006e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f75a0154f70 0x7f75a00c97d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75a0154b90 0x7f75a00c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 --2- 172.21.15.49:0/1602616331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75a00c6640 0x7f75a0155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 >> 172.21.15.49:0/1602616331 conn(0x7f75a0074840 msgr2=0x7f75a00b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 shutdown_connections 2024-05-09T08:57:30.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:30.514+0000 7f75a4f34640 1 -- 172.21.15.49:0/1602616331 wait complete. 2024-05-09T08:57:30.528 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:30.529+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:30.592 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.592 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.605 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:30.651 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:30.650+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:30.717 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.717 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.726 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:30.767 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:30.768+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:30.830 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:30.831+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:30.869 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.869 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.877 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:30.930 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:30.932+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:30.979 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:30.979 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:30.988 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:31.011 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:31.055 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:31.056+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:31.131 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:31.132+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:31.132 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:31.132 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:31.142 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:31.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.145+0000 7f7332348640 1 Processor -- start 2024-05-09T08:57:31.144 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.145+0000 7f7332348640 1 -- start start 2024-05-09T08:57:31.146 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f7332348640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f732c15b030 0x7f732c067a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f7332348640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732c0c91f0 con 0x7f732c073360 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f7332348640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732c0c8f10 con 0x7f732c15b410 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f7332348640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f732c06ef20 con 0x7f732c15b030 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f73308e5640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f732c073360 0x7f732c0c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:57734/0 (socket says 172.21.15.49:57734) 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f73308e5640 1 -- 172.21.15.49:0/3447355413 learned_addr learned my addr 172.21.15.49:0/3447355413 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732bfff640 1 --2- 172.21.15.49:0/3447355413 >> v2:172.21.15.183:3300/0 conn(0x7f732c15b030 0x7f732c067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2503808141 0 0) 0x7f732c0c91f0 con 0x7f732c073360 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7314003520 con 0x7f732c073360 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333511586 0 0) 0x7f732c0c8f10 con 0x7f732c15b410 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f732c0c91f0 con 0x7f732c15b410 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732bfff640 1 -- 172.21.15.49:0/3447355413 >> v1:172.21.15.183:6789/0 conn(0x7f732c15b410 legacy=0x7f732c154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:31.147 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732bfff640 1 -- 172.21.15.49:0/3447355413 >> v1:172.21.15.49:6789/0 conn(0x7f732c073360 legacy=0x7f732c0c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:31.148 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.147+0000 7f732bfff640 1 -- 172.21.15.49:0/3447355413 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c15b8f0 con 0x7f732c15b030 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.148+0000 7f732bfff640 1 --2- 172.21.15.49:0/3447355413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c15b030 0x7f732c067a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097a0 tx=0x7f731c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7c2f00b3f689e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.148+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c037070 con 0x7f732c15b030 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.148+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c032440 con 0x7f732c15b030 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.148+0000 7f732affd640 1 -- 172.21.15.49:0/3447355413 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c032810 con 0x7f732c15b030 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 -- 172.21.15.49:0/3447355413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c15b030 msgr2=0x7f732c067a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 --2- 172.21.15.49:0/3447355413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c15b030 0x7f732c067a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f731c0097a0 tx=0x7f731c02a270 comp rx=0 tx=0).stop 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 -- 172.21.15.49:0/3447355413 shutdown_connections 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 --2- 172.21.15.49:0/3447355413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c15b030 0x7f732c067a80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 -- 172.21.15.49:0/3447355413 >> 172.21.15.49:0/3447355413 conn(0x7f732c0c9440 msgr2=0x7f732c0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:31.149 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 -- 172.21.15.49:0/3447355413 shutdown_connections 2024-05-09T08:57:31.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.149+0000 7f7332348640 1 -- 172.21.15.49:0/3447355413 wait complete. 2024-05-09T08:57:31.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.151+0000 7f7332348640 1 Processor -- start 2024-05-09T08:57:31.150 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.151+0000 7f7332348640 1 -- start start 2024-05-09T08:57:31.152 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c073360 0x7f732c148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f732c15b030 0x7f732c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f732c15b410 0x7f732c1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f732c1552f0 con 0x7f732c15b410 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f732c06ebb0 con 0x7f732c073360 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.152+0000 7f7332348640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f732c155470 con 0x7f732c15b030 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f732c15b410 0x7f732c1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f732c15b410 0x7f732c1f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:55342/0 (socket says 172.21.15.49:55342) 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 -- 172.21.15.49:0/1011404000 learned_addr learned my addr 172.21.15.49:0/1011404000 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c073360 0x7f732c148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f732c15b030 0x7f732c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 -- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f732c15b030 msgr2=0x7f732c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f732c15b030 0x7f732c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:31.153 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 -- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c073360 msgr2=0x7f732c148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f732c073360 0x7f732c148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c1584a0 con 0x7f732c15b410 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.153+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f732c15b410 0x7f732c1f4060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f732400ca70 tx=0x7f732400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324017070 con 0x7f732c15b410 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c1f0ca0 con 0x7f732c15b410 2024-05-09T08:57:31.154 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f732400abd0 con 0x7f732c15b410 2024-05-09T08:57:31.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c1f1260 con 0x7f732c15b410 2024-05-09T08:57:31.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324012400 con 0x7f732c15b410 2024-05-09T08:57:31.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.154+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f732c04f660 con 0x7f732c15b410 2024-05-09T08:57:31.155 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.155+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7324007070 con 0x7f732c15b410 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.156+0000 7f7328ff9640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f731806be30 0x7f731806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.156+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5857+0+0 (secure 0 0 0) 0x7f732408b680 con 0x7f732c15b410 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.156+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f731806be30 0x7f731806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.156+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f732408bb10 con 0x7f732c15b410 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.157+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f732c1f14a0 0x7f732c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.157+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f732c158250 con 0x7f732c1f14a0 2024-05-09T08:57:31.156 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.157+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f732c1f14a0 0x7f732c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.157+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f731806be30 0x7f731806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f731c034000 tx=0x7f731c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.157+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f732c1f14a0 0x7f732c14a900 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.157 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.158+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c158250 con 0x7f732c1f14a0 2024-05-09T08:57:31.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.173+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c15a950 con 0x7f732c1f14a0 2024-05-09T08:57:31.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.174+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c15a950 con 0x7f732c1f14a0 2024-05-09T08:57:31.174 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:31.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.175+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f732c14f9f0 0x7f732c1457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.175+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f732c067860 con 0x7f732c14f9f0 2024-05-09T08:57:31.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.175+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f732c14f9f0 0x7f732c1457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.175+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f732c14f9f0 0x7f732c1457f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.176+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c067860 con 0x7f732c14f9f0 2024-05-09T08:57:31.200 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.201+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c0c8060 con 0x7f732c14f9f0 2024-05-09T08:57:31.203 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c0c8060 con 0x7f732c14f9f0 2024-05-09T08:57:31.203 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:31.204 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:31.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=46}) -- 0x7f732c0c7cf0 con 0x7f732c15b410 2024-05-09T08:57:31.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f732c06e5d0 con 0x7f732c15b410 2024-05-09T08:57:31.204 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7324055ad0 con 0x7f732c15b410 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f732c1534b0 0x7f732c061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.204+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f732c0621b0 con 0x7f732c1534b0 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.205+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f732c1534b0 0x7f732c061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.205+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f732c1534b0 0x7f732c061c70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.205 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.206+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c0621b0 con 0x7f732c1534b0 2024-05-09T08:57:31.233 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:31.234+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:31.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.235+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c0626c0 con 0x7f732c1534b0 2024-05-09T08:57:31.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.237+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c0626c0 con 0x7f732c1534b0 2024-05-09T08:57:31.236 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.237+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f732c0cacb0 0x7f732c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.237+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f732c062640 con 0x7f732c0cacb0 2024-05-09T08:57:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.237+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f732c0cacb0 0x7f732c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.238+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f732c0cacb0 0x7f732c0cb110 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.238+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c062640 con 0x7f732c0cacb0 2024-05-09T08:57:31.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.253+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c062640 con 0x7f732c0cacb0 2024-05-09T08:57:31.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.255+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c062640 con 0x7f732c0cacb0 2024-05-09T08:57:31.254 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.255+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f732c02c550 0x7f732c02c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.255+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f732c02ced0 con 0x7f732c02c550 2024-05-09T08:57:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.255+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f732c02c550 0x7f732c02c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.256+0000 7f732bfff640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f732c02c550 0x7f732c02c990 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.257+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c02ced0 con 0x7f732c02c550 2024-05-09T08:57:31.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.271+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c02ced0 con 0x7f732c02c550 2024-05-09T08:57:31.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.274+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c02ced0 con 0x7f732c02c550 2024-05-09T08:57:31.273 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:31.274 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:31.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.274+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f732c205440 0x7f732c207830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.274+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f732c20a600 con 0x7f732c205440 2024-05-09T08:57:31.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.274+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f732c205440 0x7f732c207830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.275+0000 7f732b7fe640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f732c205440 0x7f732c207830 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.276+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c20a600 con 0x7f732c205440 2024-05-09T08:57:31.287 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.287 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:31.288 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:31.288 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:31.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.290+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c207d70 con 0x7f732c205440 2024-05-09T08:57:31.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.292+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c207d70 con 0x7f732c205440 2024-05-09T08:57:31.291 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.292+0000 7f7332348640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f732c2083f0 0x7f732c20a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.292+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f732c20d5b0 con 0x7f732c2083f0 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.292+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f732c2083f0 0x7f732c20a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.292+0000 7f73308e5640 1 --2- 172.21.15.49:0/1011404000 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f732c2083f0 0x7f732c20a7e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:31.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.293+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f732c20d5b0 con 0x7f732c2083f0 2024-05-09T08:57:31.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:31.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.308+0000 7f7332348640 1 -- 172.21.15.49:0/1011404000 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f732c20ad20 con 0x7f732c2083f0 2024-05-09T08:57:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:31.310+0000 7f7328ff9640 1 -- 172.21.15.49:0/1011404000 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f732c20ad20 con 0x7f732c2083f0 2024-05-09T08:57:31.309 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:31.309 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:31.312 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:31.313 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:31.338 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:31.339+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:31.370 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:31.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:31.371 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-05-09T08:57:31.371 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-09T08:57:31.371 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-09T08:57:31.372 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-05-09T08:57:31.434 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:31.435+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:31.444 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.719020881999995, 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:31.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.719000283, 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:31.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.456 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.718995411999998, 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:31.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:31.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.715437112999993, 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:31.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:31.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:31.469 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.469 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.715428884999994, 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:31.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:31.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.715425076000002, 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:31.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 73.712869048000002, 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:31.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.698368240999997, 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.487 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:31.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:31.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.69835277, 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:31.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:31.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.698348026000005, 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:31.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:31.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 67.698341608999996, 2024-05-09T08:57:31.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:31.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:31.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:31.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682365390000001, 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:31.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:31.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682352688000002, 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.516 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:31.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682344770999997, 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:31.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682342263000002, 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:31.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:31.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:31.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682339798999998, 2024-05-09T08:57:31.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:31.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:31.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682324504, 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.539 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:31.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682320183000002, 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:31.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:31.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.682317042999998, 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:31.549 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:31.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:31.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 61.679857243000001, 2024-05-09T08:57:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:31.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.558 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:31.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:31.561 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.561 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-09T08:57:31.561 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:31.561 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:31.560+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:31.660 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:31.661+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:31.704 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:31.704 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:31.704 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:31.713 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:31.761 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:31.762+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:31.816 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:31.816 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:31.816 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:31.824 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:31.866 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:31.867+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:31.963 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:31.964+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:31.968 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:31.968 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.288066923000002, 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:31.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.288064055999996, 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:31.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:31.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:31.981 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.288060411000004, 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.982 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:31.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.245251245999995, 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.987 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:31.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:31.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:31.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:31.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:31.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.240459256999998, 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:31.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:31.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:31.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:31.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:31.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.237952888999999, 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:32.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:32.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:32.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:32.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.236481499000007, 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.236405661000006, 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:32.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223584251000005, 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223563252999995, 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.022 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:32.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:32.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223558858999994, 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:32.028 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:32.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.034 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223553800999994, 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.035 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:32.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223550320000001, 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:32.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:32.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.048 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223545701999996, 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:32.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:32.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223542233000003, 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:32.055 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:32.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:32.062 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223538931999997, 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:32.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:32.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:32.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.223225862999996, 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:32.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:32.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.207150040999998, 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.074 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:32.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:32.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.207145361999999, 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:32.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:32.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:32.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.207139343999998, 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.090 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:32.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:32.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.096 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:32.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:32.099 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:32.099+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:32.199 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:32.199+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:32.201 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:32.201 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:32.201 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:32.210 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:32.300 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:32.300+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:32.354 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:32.355 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:32.355 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:32.364 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:32.405 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:32.405+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:32.503 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:32.503+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:32.507 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.844064786999994, 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.514 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451408522999998, 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:32.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:32.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451360930999996, 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:32.521 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:32.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.527 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451311063999995, 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:32.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:32.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451305263999998, 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:32.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:32.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:32.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451193463999999, 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:32.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:32.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451096301999996, 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:32.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:32.552 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451090543999996, 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:32.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:32.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.451016903999999, 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:32.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:32.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:32.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.564 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.450965092000004, 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.565 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:32.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:32.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:32.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.450961372999998, 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.571 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:32.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:32.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.965904916, 2024-05-09T08:57:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:32.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:32.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.965888432, 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:32.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.965806474999994, 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:32.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:32.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.965758914999995, 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.596 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:32.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:32.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.762130471000006, 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:32.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:32.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.746051026000004, 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:32.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.746046710000002, 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:32.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:32.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:32.619 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.746037747000003, 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:32.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:32.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.625 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 62.746033687000001, 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:32.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:32.631 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:32.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:57:32.634 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:32.634+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:32.634 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:32.734 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:32.735+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:32.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.770+0000 7fa1dab9e640 1 Processor -- start 2024-05-09T08:57:32.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.770+0000 7fa1dab9e640 1 -- start start 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1dab9e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1d4152c50 0x7fa1d40c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1dab9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d40c99e0 0x7fa1d40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1dab9e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d40c6020 con 0x7fa1d4073040 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1dab9e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d40735a0 con 0x7fa1d4152c50 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1dab9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d40ca480 con 0x7fa1d40c99e0 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1d893a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1d4152c50 0x7fa1d40c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1d893a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1d4152c50 0x7fa1d40c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:53026/0 (socket says 172.21.15.49:53026) 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1d893a640 1 -- 172.21.15.49:0/1603237954 learned_addr learned my addr 172.21.15.49:0/1603237954 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1d913b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa1d4073040 0x7fa1d406f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:35890/0 (socket says 172.21.15.49:35890) 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.772+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d40c99e0 0x7fa1d40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:32.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.773+0000 7fa1d893a640 1 -- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d40c99e0 msgr2=0x7fa1d40c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.773+0000 7fa1d893a640 1 --2- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d40c99e0 0x7fa1d40c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.773+0000 7fa1d893a640 1 -- 172.21.15.49:0/1603237954 >> v1:172.21.15.183:6789/0 conn(0x7fa1d4073040 legacy=0x7fa1d406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.773+0000 7fa1d893a640 1 -- 172.21.15.49:0/1603237954 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d40ca760 con 0x7fa1d4152c50 2024-05-09T08:57:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.774+0000 7fa1d893a640 1 --2- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40c94a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009870 tx=0x7fa1c802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852fc99af3fcebf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:32.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.774+0000 7fa1d37fe640 1 -- 172.21.15.49:0/1603237954 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8037070 con 0x7fa1d4152c50 2024-05-09T08:57:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.774+0000 7fa1d37fe640 1 -- 172.21.15.49:0/1603237954 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8032640 con 0x7fa1d4152c50 2024-05-09T08:57:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.774+0000 7fa1d37fe640 1 -- 172.21.15.49:0/1603237954 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8032a10 con 0x7fa1d4152c50 2024-05-09T08:57:32.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 -- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 msgr2=0x7fa1d40c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40c94a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009870 tx=0x7fa1c802a2e0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 -- 172.21.15.49:0/1603237954 shutdown_connections 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d40c99e0 0x7fa1d40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/1603237954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40c94a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 -- 172.21.15.49:0/1603237954 >> 172.21.15.49:0/1603237954 conn(0x7fa1d40cae20 msgr2=0x7fa1d40cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 -- 172.21.15.49:0/1603237954 shutdown_connections 2024-05-09T08:57:32.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.775+0000 7fa1dab9e640 1 -- 172.21.15.49:0/1603237954 wait complete. 2024-05-09T08:57:32.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.777+0000 7fa1dab9e640 1 Processor -- start 2024-05-09T08:57:32.776 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.777+0000 7fa1dab9e640 1 -- start start 2024-05-09T08:57:32.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4073040 0x7fa1d41f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40c5b00 con 0x7fa1d40c99e0 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40c5820 con 0x7fa1d4152c50 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1dab9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d40c8720 con 0x7fa1d4073040 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53794/0 (socket says 172.21.15.49:53794) 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 -- 172.21.15.49:0/2740758692 learned_addr learned my addr 172.21.15.49:0/2740758692 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d913b640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 -- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4073040 msgr2=0x7fa1d41f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4073040 0x7fa1d41f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 -- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 msgr2=0x7fa1d40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:32.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 -- 172.21.15.49:0/2740758692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d406c7e0 con 0x7fa1d40c99e0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d913b640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.778+0000 7fa1d3fff640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9e0 tx=0x7fa1c000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.779+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0017070 con 0x7fa1d40c99e0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.779+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c000c9d0 con 0x7fa1d40c99e0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.779+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d406ca50 con 0x7fa1d40c99e0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.779+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41f3d50 con 0x7fa1d40c99e0 2024-05-09T08:57:32.780 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.779+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000cdc0 con 0x7fa1d40c99e0 2024-05-09T08:57:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.780+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b4005290 con 0x7fa1d40c99e0 2024-05-09T08:57:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.781+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1c001a070 con 0x7fa1d40c99e0 2024-05-09T08:57:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.782+0000 7fa1d17fa640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 0x7fa1b806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.782+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5857+0+0 (secure 0 0 0) 0x7fa1c008aea0 con 0x7fa1d40c99e0 2024-05-09T08:57:32.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.782+0000 7fa1d893a640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 0x7fa1b806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:32.782 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.783+0000 7fa1d893a640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 0x7fa1b806e3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8034000 tx=0x7fa1c802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:32.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:32.789+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa1c0059790 con 0x7fa1d40c99e0 2024-05-09T08:57:32.795 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:32.795 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:32.795 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:32.795 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:32.805 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:32.847 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:32.848+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:32.936 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:32.937+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:33.016 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:33.017 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:33.017 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:33.017 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:33.017 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:33.029 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:33.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.054+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fa1b4005580 con 0x7fa1d40c99e0 2024-05-09T08:57:33.053 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:33.054+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:33.138 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:33.139+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:33.145 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:33.145 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:33.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.464288547999999, 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.146 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:33.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:33.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:33.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:33.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:33.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.464282979000004, 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.154 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:33.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:33.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.161 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.464251290999997, 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.170 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.464246605, 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:33.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:33.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.177 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.464242776999995, 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.463359910999998, 2024-05-09T08:57:33.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:33.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:33.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.417787025999999, 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:33.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:33.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.417114251000001, 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:33.194 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:33.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:33.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.414393583999995, 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:33.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:33.207 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.414385542999995, 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:33.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:33.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.214 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.414381223000007, 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:33.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:33.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:33.220 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.414376918000002, 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:33.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:33.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 75.414373670000003, 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:33.227 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:33.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:33.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:33.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.232 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.399301296999994, 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:33.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:33.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:33.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:33.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:33.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:33.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.399290050000005, 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.240 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:33.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:33.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:33.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.246 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.399285629000005, 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:33.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:33.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:33.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.383181663999999, 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:33.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.383170333000002, 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:33.259 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:33.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:33.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:33.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:33.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:33.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.266 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.383162382999998, 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:33.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:33.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 63.383156505000002, 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:33.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.277 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:33.278 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:33.278 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:33.279 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:33.280 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:33.280+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:33.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:33.380 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:33.381+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.426+0000 7f192c9f9640 1 Processor -- start 2024-05-09T08:57:33.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.427+0000 7f192c9f9640 1 -- start start 2024-05-09T08:57:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192c9f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192c9f9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192c9f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1928072600 con 0x7f19280c6640 2024-05-09T08:57:33.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192c9f9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f19280728e0 con 0x7f1928154f70 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192c9f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f19280c5880 con 0x7f1928154b90 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f1925d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f1925d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53796/0 (socket says 172.21.15.49:53796) 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f1925d9b640 1 -- 172.21.15.49:0/22170271 learned_addr learned my addr 172.21.15.49:0/22170271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192659c640 1 --2- 172.21.15.49:0/22170271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192659c640 1 -- 172.21.15.49:0/22170271 >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 msgr2=0x7f19280cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192659c640 1 --2- 172.21.15.49:0/22170271 >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.428+0000 7f192659c640 1 -- 172.21.15.49:0/22170271 >> v1:172.21.15.183:6789/0 conn(0x7f19280c6640 legacy=0x7f19280c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.429+0000 7f192659c640 1 -- 172.21.15.49:0/22170271 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19280c5b60 con 0x7f1928154b90 2024-05-09T08:57:33.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.429+0000 7f1925d9b640 1 --2- 172.21.15.49:0/22170271 >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.430+0000 7f192659c640 1 --2- 172.21.15.49:0/22170271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280cb280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19180097a0 tx=0x7f191802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b55ebeabe9250c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.430+0000 7f192559a640 1 -- 172.21.15.49:0/22170271 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918037070 con 0x7f1928154b90 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.430+0000 7f192559a640 1 -- 172.21.15.49:0/22170271 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918032440 con 0x7f1928154b90 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.430+0000 7f192559a640 1 -- 172.21.15.49:0/22170271 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918032810 con 0x7f1928154b90 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 -- 172.21.15.49:0/22170271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 msgr2=0x7f19280cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 --2- 172.21.15.49:0/22170271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280cb280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19180097a0 tx=0x7f191802a230 comp rx=0 tx=0).stop 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 -- 172.21.15.49:0/22170271 shutdown_connections 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 --2- 172.21.15.49:0/22170271 >> v2:172.21.15.49:3300/0 conn(0x7f1928154f70 0x7f19280cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 --2- 172.21.15.49:0/22170271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280cb280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 -- 172.21.15.49:0/22170271 >> 172.21.15.49:0/22170271 conn(0x7f1928074840 msgr2=0x7f1928074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:33.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 -- 172.21.15.49:0/22170271 shutdown_connections 2024-05-09T08:57:33.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.431+0000 7f192c9f9640 1 -- 172.21.15.49:0/22170271 wait complete. 2024-05-09T08:57:33.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.433+0000 7f192c9f9640 1 Processor -- start 2024-05-09T08:57:33.432 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.433+0000 7f192c9f9640 1 -- start start 2024-05-09T08:57:33.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.434+0000 7f192c9f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19280c6640 0x7f19280c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19280c6640 0x7f19280c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19280c6640 0x7f19280c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53798/0 (socket says 172.21.15.49:53798) 2024-05-09T08:57:33.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192c9f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 -- 172.21.15.49:0/2686277763 learned_addr learned my addr 172.21.15.49:0/2686277763 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1928154f70 0x7f192814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f192815af40 con 0x7f19280c6640 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19280c6120 con 0x7f1928154f70 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1928071820 con 0x7f1928154b90 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1928154f70 0x7f192814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 -- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 msgr2=0x7f19280ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1928154b90 0x7f19280ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 -- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1928154f70 msgr2=0x7f192814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1928154f70 0x7f192814ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.435+0000 7f192659c640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f192814d150 con 0x7f19280c6640 2024-05-09T08:57:33.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f19280c6640 0x7f19280c78b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1918035040 tx=0x7f19180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918041020 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1918030930 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f192814d7e0 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918037040 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.436+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f192814acd0 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.437+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1928069060 con 0x7f19280c6640 2024-05-09T08:57:33.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.438+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1918044020 con 0x7f19280c6640 2024-05-09T08:57:33.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.439+0000 7f1916ffd640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f190806be30 0x7f190806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.439+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5857+0+0 (secure 0 0 0) 0x7f19180ab5e0 con 0x7f19280c6640 2024-05-09T08:57:33.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.439+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f1928069060 con 0x7f19280c6640 2024-05-09T08:57:33.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f190806be30 0x7f190806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f19281539d0 0x7f19281505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f192804f660 con 0x7f19281539d0 2024-05-09T08:57:33.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f19281539d0 0x7f19281505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f190806be30 0x7f190806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1928151b60 tx=0x7f191c009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.440+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f19281539d0 0x7f19281505d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.440 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:33.441 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:33.441 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:33.441 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:33.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.442+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f192804f660 con 0x7f19281539d0 2024-05-09T08:57:33.450 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:33.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.471+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f192814ed40 con 0x7f19281539d0 2024-05-09T08:57:33.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.474+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f192814ed40 con 0x7f19281539d0 2024-05-09T08:57:33.473 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:33.473 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.474+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1928153db0 0x7f192814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.474+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1928152f70 con 0x7f1928153db0 2024-05-09T08:57:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.474+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1928153db0 0x7f192814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.475+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f1928153db0 0x7f192814b3b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.475+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1928152f70 con 0x7f1928153db0 2024-05-09T08:57:33.480 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:33.481+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:33.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.501+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f192814a1d0 con 0x7f1928153db0 2024-05-09T08:57:33.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.504+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f192814a1d0 con 0x7f1928153db0 2024-05-09T08:57:33.503 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.504+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=46}) -- 0x7f1928151900 con 0x7f19280c6640 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.504+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f192814a630 con 0x7f19280c6640 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.504+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f19180b2070 con 0x7f19280c6640 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.505+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f19281f46a0 0x7f19281f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.505+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f19280cbde0 con 0x7f19281f46a0 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.505+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f19281f46a0 0x7f19281f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.505+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f19281f46a0 0x7f19281f4af0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.506+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19280cbde0 con 0x7f19281f46a0 2024-05-09T08:57:33.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.536+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f19280cc2f0 con 0x7f19281f46a0 2024-05-09T08:57:33.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.538+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f19280cc2f0 con 0x7f19281f46a0 2024-05-09T08:57:33.538 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.539+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1928061c70 0x7f19280620d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.539+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f19280cc2f0 con 0x7f1928061c70 2024-05-09T08:57:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.539+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1928061c70 0x7f19280620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.539+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f1928061c70 0x7f19280620d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.540+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19280cc2f0 con 0x7f1928061c70 2024-05-09T08:57:33.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.569+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f19280cc270 con 0x7f1928061c70 2024-05-09T08:57:33.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.572+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f19280cc270 con 0x7f1928061c70 2024-05-09T08:57:33.571 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:33.571 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.572+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f192802c040 0x7f192802c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.572+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f192802c9a0 con 0x7f192802c040 2024-05-09T08:57:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.572+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f192802c040 0x7f192802c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.573+0000 7f192659c640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f192802c040 0x7f192802c460 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.573+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f192802c9a0 con 0x7f192802c040 2024-05-09T08:57:33.582 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:33.582+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:33.592 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:33.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:33.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.602+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f192802c9a0 con 0x7f192802c040 2024-05-09T08:57:33.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.604+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f192802c9a0 con 0x7f192802c040 2024-05-09T08:57:33.603 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:33.604 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:33.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.604+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f192802ceb0 0x7f1928207410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.604+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f192802d570 con 0x7f192802ceb0 2024-05-09T08:57:33.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.605+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f192802ceb0 0x7f1928207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.605+0000 7f1925d9b640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f192802ceb0 0x7f1928207410 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.606+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f192802d570 con 0x7f192802ceb0 2024-05-09T08:57:33.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.623+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1928207950 con 0x7f192802ceb0 2024-05-09T08:57:33.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.625+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1928207950 con 0x7f192802ceb0 2024-05-09T08:57:33.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.626+0000 7f192c9f9640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1928208070 0x7f19282084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:33.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.626+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f192820d450 con 0x7f1928208070 2024-05-09T08:57:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.626+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1928208070 0x7f19282084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.627+0000 7f1926d9d640 1 --2- 172.21.15.49:0/2686277763 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1928208070 0x7f19282084d0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:33.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.627+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f192820d450 con 0x7f1928208070 2024-05-09T08:57:33.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.645+0000 7f192c9f9640 1 -- 172.21.15.49:0/2686277763 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f192820c250 con 0x7f1928208070 2024-05-09T08:57:33.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.647+0000 7f1916ffd640 1 -- 172.21.15.49:0/2686277763 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f192820c250 con 0x7f1928208070 2024-05-09T08:57:33.647 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:33.647 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:33.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.648+0000 7fa1d17fa640 1 -- 172.21.15.49:0/2740758692 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v46) ==== 97+0+0 (secure 0 0 0) 0x7fa1c0059090 con 0x7fa1d40c99e0 2024-05-09T08:57:33.651 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:57:33.653 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:33.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 msgr2=0x7fa1b806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 0x7fa1b806e3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8034000 tx=0x7fa1c802c000 comp rx=0 tx=0).stop 2024-05-09T08:57:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 msgr2=0x7fa1d41f0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:33.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000e9e0 tx=0x7fa1c000eea0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 shutdown_connections 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fa1b806bf30 0x7fa1b806e3f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1d4152c50 0x7fa1d40cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fa1d40c99e0 0x7fa1d41f0460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 --2- 172.21.15.49:0/2740758692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1d4073040 0x7fa1d41f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 >> 172.21.15.49:0/2740758692 conn(0x7fa1d40cae20 msgr2=0x7fa1d4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 shutdown_connections 2024-05-09T08:57:33.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:33.654+0000 7fa1dab9e640 1 -- 172.21.15.49:0/2740758692 wait complete. 2024-05-09T08:57:33.682 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:33.683+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:33.726 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:33.726 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:33.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:33.783 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:33.784+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:33.850 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:33.851 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:33.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:33.904 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:33.905+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:33.972 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:33.982 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:33.985 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:33.985+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:34.086 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:34.086+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:34.134 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.134 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.147 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:34.191 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:34.191+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:34.261 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.262 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:34.312 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:34.312+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:34.388 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:34.389+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:34.414 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.414 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.422 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:34.489 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:34.489+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:34.525 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.525 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.534 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:34.590 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:34.591+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:34.678 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.679 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:34.690 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:34.690+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:34.791 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.791 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.792 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:34.792+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:34.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:34.892 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:34.893+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:34.902 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:34.902 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:34.911 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:34.993 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:34.993+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:35.014 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:35.452 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:35.454 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:35.454 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:35.454+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:35.555 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:35.555+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:35.600 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:35.601 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:35.610 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:35.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:35.656 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:35.657+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:35.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:57:35.753 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:35.753 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:35.762 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:35.757+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:35.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.765+0000 7fdfc30f7640 1 Processor -- start 2024-05-09T08:57:35.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.765+0000 7fdfc30f7640 1 -- start start 2024-05-09T08:57:35.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc30f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfbc154b90 0x7fdfbc0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc30f7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 0x7fdfbc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfbbfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fdfbc154f70 0x7fdfbc0c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41650/0 (socket says 172.21.15.49:41650) 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc1694640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 0x7fdfbc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc0e93640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfbc154b90 0x7fdfbc0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc1694640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 0x7fdfbc0cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53802/0 (socket says 172.21.15.49:53802) 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfbbfff640 1 -- 172.21.15.49:0/1441757888 learned_addr learned my addr 172.21.15.49:0/1441757888 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.766+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfbc072600 con 0x7fdfbc154f70 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fdfbc0728e0 con 0x7fdfbc0c6d50 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc0c5880 con 0x7fdfbc154b90 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228294230 0 0) 0x7fdfbc072600 con 0x7fdfbc154f70 2024-05-09T08:57:35.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfac003520 con 0x7fdfbc154f70 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 737536680 0 0) 0x7fdfac003520 con 0x7fdfbc154f70 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfbc154b90 msgr2=0x7fdfbc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfbc154b90 0x7fdfbc0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 msgr2=0x7fdfbc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 --2- 172.21.15.49:0/1441757888 >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 0x7fdfbc0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc0c5b60 con 0x7fdfbc154f70 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fdfa8001670 con 0x7fdfbc154f70 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 >> v1:172.21.15.49:6789/0 conn(0x7fdfbc154f70 legacy=0x7fdfbc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfac0039b0 0x7fdfac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac004800 0x7fdfac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfac0051c0 0x7fdfac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0728e0 con 0x7fdfac0051c0 2024-05-09T08:57:35.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0c5880 con 0x7fdfac0039b0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac00bf00 con 0x7fdfac004800 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.767+0000 7fdfc1694640 1 --2- 172.21.15.49:0/1441757888 >> v2:172.21.15.49:3300/0 conn(0x7fdfbc0c6d50 0x7fdfbc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfac0051c0 0x7fdfac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 -- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac004800 msgr2=0x7fdfac004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac004800 0x7fdfac004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 -- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfac0039b0 msgr2=0x7fdfac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfac0039b0 0x7fdfac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.768+0000 7fdfc1694640 1 -- 172.21.15.49:0/1441757888 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc0c5b60 con 0x7fdfac0051c0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.769+0000 7fdfc1694640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfac0051c0 0x7fdfac00b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009d80 tx=0x7fdfac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b47eabfafea7923 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.769+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4009dc0 con 0x7fdfac0051c0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.769+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb4033dc0 con 0x7fdfac0051c0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.769+0000 7fdfbb7fe640 1 -- 172.21.15.49:0/1441757888 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4003b90 con 0x7fdfac0051c0 2024-05-09T08:57:35.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfac0051c0 msgr2=0x7fdfac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/1441757888 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfac0051c0 0x7fdfac00b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009d80 tx=0x7fdfac005830 comp rx=0 tx=0).stop 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc1694640 1 -- 172.21.15.49:0/1441757888 reap_dead start 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 shutdown_connections 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 >> 172.21.15.49:0/1441757888 conn(0x7fdfbc074840 msgr2=0x7fdfbc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 shutdown_connections 2024-05-09T08:57:35.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.770+0000 7fdfc30f7640 1 -- 172.21.15.49:0/1441757888 wait complete. 2024-05-09T08:57:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.772+0000 7fdfc30f7640 1 Processor -- start 2024-05-09T08:57:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.772+0000 7fdfc30f7640 1 -- start start 2024-05-09T08:57:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc30f7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac0051c0 0x7fdfbc148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.777 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc30f7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0b4cd0 0x7fdfbc0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc0e93640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac0051c0 0x7fdfbc148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc0e93640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac0051c0 0x7fdfbc148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36632/0 (socket says 172.21.15.49:36632) 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0b4cd0 0x7fdfbc0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0b4cd0 0x7fdfbc0b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53822/0 (socket says 172.21.15.49:53822) 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc0e93640 1 -- 172.21.15.49:0/381848547 learned_addr learned my addr 172.21.15.49:0/381848547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.774+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfbc0ce180 0x7fdfbc0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc15af40 con 0x7fdfbc0b4cd0 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0c6120 con 0x7fdfbc0ce180 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc0cb280 con 0x7fdfac0051c0 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfbc0ce180 0x7fdfbc0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 -- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfbc0ce180 msgr2=0x7fdfbc0b59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.778 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfbc0ce180 0x7fdfbc0b59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 -- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0b4cd0 msgr2=0x7fdfbc0b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fdfbc0b4cd0 0x7fdfbc0b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc0ce750 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdfac0051c0 0x7fdfbc148b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0004650 tx=0x7fdfb000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.775+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0017070 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.776+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc1efd00 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.776+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb000c8c0 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.776+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000cc90 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.776+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1f02c0 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.777+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fdfbc04f660 con 0x7fdfac0051c0 2024-05-09T08:57:35.779 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.778+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfb001a070 con 0x7fdfac0051c0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.779+0000 7fdfb97fa640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdfa006be30 0x7fdfa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.779+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5857+0+0 (secure 0 0 0) 0x7fdfb008b560 con 0x7fdfac0051c0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.779+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdfa006be30 0x7fdfa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.779+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v46) ==== 54+0+16 (secure 0 0 0) 0x7fdfb008b910 con 0x7fdfac0051c0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.780+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fdfbc14ad10 0x7fdfbc14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.780+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc1f0610 con 0x7fdfbc14ad10 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.780+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fdfa006be30 0x7fdfa006e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0cecd0 tx=0x7fdfa8001b20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.780+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fdfbc14ad10 0x7fdfbc14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.780+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fdfbc14ad10 0x7fdfbc14b0f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.781 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.781+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc1f0610 con 0x7fdfbc14ad10 2024-05-09T08:57:35.795 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.796+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc158920 con 0x7fdfbc14ad10 2024-05-09T08:57:35.797 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.798+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc158920 con 0x7fdfbc14ad10 2024-05-09T08:57:35.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.798+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fdfbc141ba0 0x7fdfbc141fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.798+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc14bb40 con 0x7fdfbc141ba0 2024-05-09T08:57:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.798+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fdfbc141ba0 0x7fdfbc141fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.798 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.799+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fdfbc141ba0 0x7fdfbc141fc0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.799+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc14bb40 con 0x7fdfbc141ba0 2024-05-09T08:57:35.814 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.815+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc074fa0 con 0x7fdfbc141ba0 2024-05-09T08:57:35.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc074fa0 con 0x7fdfbc141ba0 2024-05-09T08:57:35.816 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=47}) -- 0x7fdfbc074cc0 con 0x7fdfac0051c0 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdfbc142500 con 0x7fdfac0051c0 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdfb0021040 con 0x7fdfac0051c0 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fdfbc062180 0x7fdfbc062590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.817+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc0ccf40 con 0x7fdfbc062180 2024-05-09T08:57:35.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.818+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fdfbc062180 0x7fdfbc062590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.818+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fdfbc062180 0x7fdfbc062590 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.819+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc0ccf40 con 0x7fdfbc062180 2024-05-09T08:57:35.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.833+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc0cbdc0 con 0x7fdfbc062180 2024-05-09T08:57:35.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.835+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc0cbdc0 con 0x7fdfbc062180 2024-05-09T08:57:35.834 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:35.835 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:35.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.835+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fdfbc0cc8a0 0x7fdfbc02c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.835+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc0cbdc0 con 0x7fdfbc0cc8a0 2024-05-09T08:57:35.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.836+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fdfbc0cc8a0 0x7fdfbc02c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.836+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fdfbc0cc8a0 0x7fdfbc02c040 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.837+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc0cbdc0 con 0x7fdfbc0cc8a0 2024-05-09T08:57:35.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.856+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc0cbdc0 con 0x7fdfbc0cc8a0 2024-05-09T08:57:35.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.858+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc0cbdc0 con 0x7fdfbc0cc8a0 2024-05-09T08:57:35.858 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:35.858 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:35.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.858+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fdfbc02ce50 0x7fdfbc2010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.858+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc0cbdc0 con 0x7fdfbc02ce50 2024-05-09T08:57:35.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.859+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fdfbc02ce50 0x7fdfbc2010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.859+0000 7fdfc0e93640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fdfbc02ce50 0x7fdfbc2010b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.860+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc0cbdc0 con 0x7fdfbc02ce50 2024-05-09T08:57:35.859 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:35.858+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:35.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.862+0000 7f861cb10640 1 Processor -- start 2024-05-09T08:57:35.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.862+0000 7f861cb10640 1 -- start start 2024-05-09T08:57:35.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861cb10640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618074ab0 0x7f86180c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861cb10640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8618074e90 0x7f861815e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861cb10640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86180c9570 con 0x7f8618073330 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861cb10640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f86180c8e00 con 0x7f8618074ab0 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861cb10640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f86180c9850 con 0x7f8618074e90 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618074ab0 0x7f86180c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8618074ab0 0x7f86180c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53836/0 (socket says 172.21.15.49:53836) 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f861659c640 1 -- 172.21.15.49:0/2175746156 learned_addr learned my addr 172.21.15.49:0/2175746156 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.864+0000 7f8615d9b640 1 --2- 172.21.15.49:0/2175746156 >> v2:172.21.15.183:3300/0 conn(0x7f8618074e90 0x7f861815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.865+0000 7f861659c640 1 -- 172.21.15.49:0/2175746156 >> v2:172.21.15.183:3300/0 conn(0x7f8618074e90 msgr2=0x7f861815e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.865+0000 7f861659c640 1 --2- 172.21.15.49:0/2175746156 >> v2:172.21.15.183:3300/0 conn(0x7f8618074e90 0x7f861815e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.865+0000 7f861659c640 1 -- 172.21.15.49:0/2175746156 >> v1:172.21.15.183:6789/0 conn(0x7f8618073330 legacy=0x7f8618154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.865+0000 7f861559a640 1 -- 172.21.15.49:0/2175746156 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1885888535 0 0) 0x7f86180c9570 con 0x7f8618073330 2024-05-09T08:57:35.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.865+0000 7f861659c640 1 -- 172.21.15.49:0/2175746156 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86180711e0 con 0x7f8618074ab0 2024-05-09T08:57:35.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.866+0000 7f861659c640 1 --2- 172.21.15.49:0/2175746156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f86180c88c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8618075920 tx=0x7f860802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786343819867937 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.866+0000 7f861559a640 1 -- 172.21.15.49:0/2175746156 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608037070 con 0x7f8618074ab0 2024-05-09T08:57:35.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.866+0000 7f861559a640 1 -- 172.21.15.49:0/2175746156 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8608032440 con 0x7f8618074ab0 2024-05-09T08:57:35.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.866+0000 7f861559a640 1 -- 172.21.15.49:0/2175746156 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608032810 con 0x7f8618074ab0 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 -- 172.21.15.49:0/2175746156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 msgr2=0x7f86180c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 --2- 172.21.15.49:0/2175746156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f86180c88c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8618075920 tx=0x7f860802a270 comp rx=0 tx=0).stop 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 -- 172.21.15.49:0/2175746156 shutdown_connections 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 --2- 172.21.15.49:0/2175746156 >> v2:172.21.15.183:3300/0 conn(0x7f8618074e90 0x7f861815e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 --2- 172.21.15.49:0/2175746156 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f86180c88c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.867+0000 7f861cb10640 1 -- 172.21.15.49:0/2175746156 >> 172.21.15.49:0/2175746156 conn(0x7f861806b200 msgr2=0x7f861806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.868+0000 7f861cb10640 1 -- 172.21.15.49:0/2175746156 shutdown_connections 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.868+0000 7f861cb10640 1 -- 172.21.15.49:0/2175746156 wait complete. 2024-05-09T08:57:35.868 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:35.869 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:35.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.870+0000 7f861cb10640 1 Processor -- start 2024-05-09T08:57:35.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.870+0000 7f861cb10640 1 -- start start 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861659c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f8615d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861659c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36644/0 (socket says 172.21.15.49:36644) 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f8615d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53848/0 (socket says 172.21.15.49:53848) 2024-05-09T08:57:35.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861659c640 1 -- 172.21.15.49:0/3522969508 learned_addr learned my addr 172.21.15.49:0/3522969508 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8618074e90 0x7f86181f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86180c9b30 con 0x7f8618074ab0 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86180717a0 con 0x7f8618074e90 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.871+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8618071cc0 con 0x7f8618073330 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f8616d9d640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8618074e90 0x7f86181f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 -- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8618074e90 msgr2=0x7f86181f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8618074e90 0x7f86181f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 -- 172.21.15.49:0/3522969508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 msgr2=0x7f861814fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8618150350 con 0x7f8618073330 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f8615d9b640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.872+0000 7f861659c640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8608009960 tx=0x7f86080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.873+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608037040 con 0x7f8618073330 2024-05-09T08:57:35.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.873+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8608042070 con 0x7f8618073330 2024-05-09T08:57:35.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.873+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608030960 con 0x7f8618073330 2024-05-09T08:57:35.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.873+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8618153700 con 0x7f8618073330 2024-05-09T08:57:35.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.874+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181f1580 con 0x7f8618073330 2024-05-09T08:57:35.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.875+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861804f660 con 0x7f8618073330 2024-05-09T08:57:35.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.876+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8608030b00 con 0x7f8618073330 2024-05-09T08:57:35.876 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:35.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.877+0000 7f8606ffd640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 0x7f85f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.877+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5857+0+0 (secure 0 0 0) 0x7f86080ab710 con 0x7f8618073330 2024-05-09T08:57:35.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.877+0000 7f8615d9b640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 0x7f85f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.878+0000 7f8615d9b640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 0x7f85f806e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8618153ca0 tx=0x7f860c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.879+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc0cbdc0 con 0x7fdfbc02ce50 2024-05-09T08:57:35.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.881+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc0cbdc0 con 0x7fdfbc02ce50 2024-05-09T08:57:35.880 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:35.881 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:35.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.881+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fdfbc02d2b0 0x7fdfbc203b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.881+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc0ccd00 con 0x7fdfbc02d2b0 2024-05-09T08:57:35.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.881+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8608079f80 con 0x7f8618073330 2024-05-09T08:57:35.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.882+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fdfbc02d2b0 0x7fdfbc203b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.882+0000 7fdfbbfff640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fdfbc02d2b0 0x7fdfbc203b10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.883+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc0ccd00 con 0x7fdfbc02d2b0 2024-05-09T08:57:35.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.897+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc0ccd00 con 0x7fdfbc02d2b0 2024-05-09T08:57:35.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.899+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc0ccd00 con 0x7fdfbc02d2b0 2024-05-09T08:57:35.898 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.899+0000 7fdfc30f7640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fdfbc204560 0x7fdfbc206950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.899+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fdfbc209720 con 0x7fdfbc204560 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.899+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fdfbc204560 0x7fdfbc206950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.900+0000 7fdfc1694640 1 --2- 172.21.15.49:0/381848547 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fdfbc204560 0x7fdfbc206950 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:35.899 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.900+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfbc209720 con 0x7fdfbc204560 2024-05-09T08:57:35.914 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.915+0000 7fdfc30f7640 1 -- 172.21.15.49:0/381848547 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdfbc206e90 con 0x7fdfbc204560 2024-05-09T08:57:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:35.917+0000 7fdfb97fa640 1 -- 172.21.15.49:0/381848547 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdfbc206e90 con 0x7fdfbc204560 2024-05-09T08:57:35.916 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:35.916 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:35.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:35.920 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:35.961 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:35.962+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:36.019 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:36.019 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:36.019 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:36.019 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:36.019 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:36.027 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:36.069 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:36.069+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:36.132 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.133+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f8618155140 con 0x7f8618073330 2024-05-09T08:57:36.160 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:36.161+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.508856422999997, 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:36.173 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:36.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:36.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.116200159000002, 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.180 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:36.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:36.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.116152567, 2024-05-09T08:57:36.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:36.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:36.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.116102699999999, 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.193 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:36.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:36.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.116096900000002, 2024-05-09T08:57:36.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:36.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:36.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:36.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.115985100000003, 2024-05-09T08:57:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:36.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:36.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:36.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:36.211 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.115887938, 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:36.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:36.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.217 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.11588218, 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:36.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.223 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.115808540000003, 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:36.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.115756727999994, 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:36.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:36.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.115753009000002, 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.236 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:36.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:36.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.630696552000003, 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.242 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:36.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.247 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.630680068000004, 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.248 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:36.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.253 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.630598110999998, 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.254 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:36.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:36.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.630550550999999, 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:36.260 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:36.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:36.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.426922106999996, 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.266 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:36.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:36.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.410842661999993, 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:36.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:36.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:36.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.410838346000006, 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:36.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:36.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.410829383000006, 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:36.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:36.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.410825322999997, 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:36.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:36.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:36.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:36.294 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:36.296 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:36.296 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:36.296+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:36.396 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:36.398+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:36.412 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:36.412 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:36.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:36.507 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:36.508+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:36.562 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:36.562 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:36.563 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-09T08:57:36.563 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-05-09T08:57:36.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-09T08:57:36.578 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:36.579 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:36.579 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:36.579 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:36.579 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:36.586 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:36.599 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:36.598+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.664+0000 7f8606ffd640 1 -- 172.21.15.49:0/3522969508 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v47) ==== 91+0+0 (secure 0 0 0) 0x7f8608079880 con 0x7f8618073330 2024-05-09T08:57:36.664 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 msgr2=0x7f85f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 0x7f85f806e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8618153ca0 tx=0x7f860c009190 comp rx=0 tx=0).stop 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 msgr2=0x7f861814f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8608009960 tx=0x7f86080046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 shutdown_connections 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f85f806be60 0x7f85f806e320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8618074e90 0x7f86181f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8618074ab0 0x7f861814fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 --2- 172.21.15.49:0/3522969508 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8618073330 0x7f861814f8d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 >> 172.21.15.49:0/3522969508 conn(0x7f861806b200 msgr2=0x7f86180ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:36.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 shutdown_connections 2024-05-09T08:57:36.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.667+0000 7f861cb10640 1 -- 172.21.15.49:0/3522969508 wait complete. 2024-05-09T08:57:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.700+0000 7fd78ed61640 1 Processor -- start 2024-05-09T08:57:36.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.700+0000 7fd78ed61640 1 -- start start 2024-05-09T08:57:36.700 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:36.701+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.701+0000 7fd78ed61640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 0x7fd7880c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.701+0000 7fd78ed61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.701+0000 7fd78ed61640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7880c6020 con 0x7fd788073040 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd78ed61640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd7880735a0 con 0x7fd788152c50 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd78cafd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 0x7fd7880c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36658/0 (socket says 172.21.15.49:36658) 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 -- 172.21.15.49:0/2193771796 learned_addr learned my addr 172.21.15.49:0/2193771796 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd78cafd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 0x7fd7880c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53852/0 (socket says 172.21.15.49:53852) 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880ca480 con 0x7fd7880c99e0 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd7877fe640 1 -- 172.21.15.49:0/2193771796 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131191075 0 0) 0x7fd7880c6020 con 0x7fd788073040 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd7877fe640 1 -- 172.21.15.49:0/2193771796 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd778003520 con 0x7fd788073040 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 -- 172.21.15.49:0/2193771796 >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 msgr2=0x7fd7880c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 --2- 172.21.15.49:0/2193771796 >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 0x7fd7880c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 -- 172.21.15.49:0/2193771796 >> v1:172.21.15.183:6789/0 conn(0x7fd788073040 legacy=0x7fd78806f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.702+0000 7fd787fff640 1 -- 172.21.15.49:0/2193771796 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7880ca760 con 0x7fd7880c99e0 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.703+0000 7fd787fff640 1 --2- 172.21.15.49:0/2193771796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd774009870 tx=0x7fd77402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a303d82d96be7a0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.703+0000 7fd7877fe640 1 -- 172.21.15.49:0/2193771796 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774037070 con 0x7fd7880c99e0 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.703+0000 7fd7877fe640 1 -- 172.21.15.49:0/2193771796 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77402ae50 con 0x7fd7880c99e0 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.703+0000 7fd7877fe640 1 -- 172.21.15.49:0/2193771796 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774032860 con 0x7fd7880c99e0 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 msgr2=0x7fd7880c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 --2- 172.21.15.49:0/2193771796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd774009870 tx=0x7fd77402a300 comp rx=0 tx=0).stop 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 shutdown_connections 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 --2- 172.21.15.49:0/2193771796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd7880c99e0 0x7fd7880c75c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 --2- 172.21.15.49:0/2193771796 >> v2:172.21.15.49:3300/0 conn(0x7fd788152c50 0x7fd7880c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 >> 172.21.15.49:0/2193771796 conn(0x7fd7880cae20 msgr2=0x7fd7880cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 shutdown_connections 2024-05-09T08:57:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.704+0000 7fd78ed61640 1 -- 172.21.15.49:0/2193771796 wait complete. 2024-05-09T08:57:36.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.705+0000 7fd78ed61640 1 Processor -- start 2024-05-09T08:57:36.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.706+0000 7fd78ed61640 1 -- start start 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.706+0000 7fd78ed61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 0x7fd7881ecb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.706+0000 7fd78ed61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7880c99e0 0x7fd788151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78ed61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78ed61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd7880c5b00 con 0x7fd788152c50 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78ed61640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7880c5820 con 0x7fd7880c99e0 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78ed61640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd7880c8720 con 0x7fd788073040 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd787fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7880c99e0 0x7fd788151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78cafd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 0x7fd7881ecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78cafd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 0x7fd7881ecb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36668/0 (socket says 172.21.15.49:36668) 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78cafd640 1 -- 172.21.15.49:0/489080951 learned_addr learned my addr 172.21.15.49:0/489080951 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53866/0 (socket says 172.21.15.49:53866) 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 -- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 msgr2=0x7fd7881ecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 0x7fd7881ecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 -- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7880c99e0 msgr2=0x7fd788151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7880c99e0 0x7fd788151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.707+0000 7fd78d2fe640 1 -- 172.21.15.49:0/489080951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7880cd760 con 0x7fd788152c50 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.708+0000 7fd78d2fe640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c700 tx=0x7fd78000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.708+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780003b70 con 0x7fd788152c50 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.708+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd780003d30 con 0x7fd788152c50 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.709+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780005310 con 0x7fd788152c50 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.709+0000 7fd78ed61640 1 -- 172.21.15.49:0/489080951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7880cda20 con 0x7fd788152c50 2024-05-09T08:57:36.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.709+0000 7fd78ed61640 1 -- 172.21.15.49:0/489080951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7880cdfe0 con 0x7fd788152c50 2024-05-09T08:57:36.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.710+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78804f660 con 0x7fd788152c50 2024-05-09T08:57:36.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.711+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7800054b0 con 0x7fd788152c50 2024-05-09T08:57:36.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.712+0000 7fd7857fa640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:36.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.712+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5857+0+0 (secure 0 0 0) 0x7fd78008ab70 con 0x7fd788152c50 2024-05-09T08:57:36.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.712+0000 7fd78cafd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:36.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.713+0000 7fd78cafd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 0x7fd76c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd77c015230 tx=0x7fd77c00f040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:36.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:36.720+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd780058ec0 con 0x7fd788152c50 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.082563780000001, 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:36.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.768 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.082558211000006, 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:36.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:36.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:36.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.082526522999999, 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:36.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.082521837000002, 2024-05-09T08:57:36.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:36.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:36.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:36.787 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.082518008999998, 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:36.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.791 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.081635143, 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:36.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.036062258000001, 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:36.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:36.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.035389483000003, 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:36.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.032668815999997, 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:36.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:36.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:36.812 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.032660774999997, 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:36.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:36.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.032656454999994, 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.818 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:36.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.032652150000004, 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:36.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.829 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.032648902000005, 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:36.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.833 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.017576528999996, 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:36.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.839 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.017565282000007, 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:36.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:36.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.017560861000007, 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:36.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:36.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.001456895999993, 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.851 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:36.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:36.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.001445564999997, 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:36.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:36.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.001437615, 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:36.862 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:36.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:36.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.001431737000004, 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:36.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:36.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:36.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:36.872 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:36.873 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:36.874 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:36.874+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:36.974 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:36.976+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:37.029 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.029+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7fd7880c8280 con 0x7fd788152c50 2024-05-09T08:57:37.030 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.031 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:37.031 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:37.031 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:37.076 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:37.077+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:37.177 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:37.178+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:37.192 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.193 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:37.193 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:37.193 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:37.193 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.202 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:37.278 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:37.279+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.674418677000006, 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:37.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:37.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.674394324999994, 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:37.362 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:37.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:37.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.674388570000005, 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:37.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:37.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.674373338999999, 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:37.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:37.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:37.379 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.631035354000005, 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:37.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.631027392999997, 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:37.385 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:37.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:37.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.623819363999999, 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:37.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.396 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 79.623808619000002, 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:37.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:37.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609834046000003, 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:37.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:37.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609823937000002, 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:37.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609821299000004, 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:37.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:37.419 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609805696999999, 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.420 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:37.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:37.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609803048000003, 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:37.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:37.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.431 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609788922000007, 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:37.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609786643000007, 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:37.437 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:37.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:37.442 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609778738000003, 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:37.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:37.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 73.609777217000001, 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:37.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:37.451 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.589701367999993, 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:37.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.589679907999994, 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:37.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 67.589672902999993, 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:37.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.467 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.468 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:37.469 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:37.469+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:37.569 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:37.570+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:37.602 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:37.602 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.612 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:37.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.661+0000 7fd7857fa640 1 -- 172.21.15.49:0/489080951 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v48) ==== 133+0+0 (secure 0 0 0) 0x7fd78005d1c0 con 0x7fd788152c50 2024-05-09T08:57:37.660 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-09T08:57:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.665+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 msgr2=0x7fd76c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.665+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 0x7fd76c06e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd77c015230 tx=0x7fd77c00f040 comp rx=0 tx=0).stop 2024-05-09T08:57:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.665+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 msgr2=0x7fd788151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:37.664 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.665+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd78000c700 tx=0x7fd78000cbc0 comp rx=0 tx=0).stop 2024-05-09T08:57:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 shutdown_connections 2024-05-09T08:57:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd76c06be30 0x7fd76c06e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd788152c50 0x7fd788151f40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7880c99e0 0x7fd788151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:37.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 --2- 172.21.15.49:0/489080951 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd788073040 0x7fd7881ecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:37.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.668+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 >> 172.21.15.49:0/489080951 conn(0x7fd7880cae20 msgr2=0x7fd7880c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.670+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 shutdown_connections 2024-05-09T08:57:37.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:37.670+0000 7fd772ffd640 1 -- 172.21.15.49:0/489080951 wait complete. 2024-05-09T08:57:37.670 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:37.671+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:37.730 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.730 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:37.730 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:37.730 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:37.730 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:37.771 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:37.772+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:37.855 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.173675336000002, 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.173655365000002, 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:37.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:37.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:37.865 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.172826599000004, 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:37.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:37.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.171823707000001, 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:37.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.171520876000002, 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:37.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:37.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.130334317999996, 2024-05-09T08:57:37.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:37.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:37.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:37.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.130322292000002, 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:37.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:37.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.130308822000003, 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.894 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:37.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:37.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.129284420000005, 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:37.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:37.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:37.904 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.128857131000004, 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:37.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.909 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.128740790999998, 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:37.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:37.914 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.127466795000004, 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:37.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.918 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.125717394999995, 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:37.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:37.924 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.125389251000001, 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:37.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.929 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.115260622999998, 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:37.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:37.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.124517621999999, 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:37.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:37.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.124508235999997, 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:37.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:37.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:37.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:37.955 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.124497192000007, 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:37.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:37.960 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.122762195999996, 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:37.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 68.092857989999999, 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:37.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:37.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:37.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:37.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:37.972 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:37.973 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:37.974 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:37.975+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:37.975 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:38.075 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:38.076+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:38.089 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:38.090 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:38.090 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:38.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:38.172 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.173+0000 7f8da7e61640 1 Processor -- start 2024-05-09T08:57:38.173 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.173+0000 7f8da7e61640 1 -- start start 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da7e61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 0x7f8da015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da7e61640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8da015b270 0x7f8da00c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da7e61640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8da015f190 con 0x7f8da0154ec0 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da7e61640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8da00c8410 con 0x7f8da015b270 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da5bfd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8da0154ec0 0x7f8da006e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41660/0 (socket says 172.21.15.49:41660) 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da5bfd640 1 -- 172.21.15.49:0/572814792 learned_addr learned my addr 172.21.15.49:0/572814792 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:38.175 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8da0071220 con 0x7f8da01552a0 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da53fc640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 0x7f8da015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da63fe640 1 --2- 172.21.15.49:0/572814792 >> v2:172.21.15.183:3300/0 conn(0x7f8da015b270 0x7f8da00c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da4bfb640 1 -- 172.21.15.49:0/572814792 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209451739 0 0) 0x7f8da015f190 con 0x7f8da0154ec0 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.175+0000 7f8da4bfb640 1 -- 172.21.15.49:0/572814792 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d8c003520 con 0x7f8da0154ec0 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.176+0000 7f8da63fe640 1 -- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 msgr2=0x7f8da015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.176+0000 7f8da63fe640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 0x7f8da015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.176+0000 7f8da63fe640 1 -- 172.21.15.49:0/572814792 >> v1:172.21.15.49:6789/0 conn(0x7f8da0154ec0 legacy=0x7f8da006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.176+0000 7f8da63fe640 1 -- 172.21.15.49:0/572814792 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da00c8130 con 0x7f8da015b270 2024-05-09T08:57:38.176 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.176+0000 7f8da53fc640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 0x7f8da015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.177+0000 7f8da63fe640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da015b270 0x7f8da00c9da0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009dc0 tx=0x7f8d9802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b8d8abd6511726 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.177+0000 7f8da4bfb640 1 -- 172.21.15.49:0/572814792 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98007740 con 0x7f8da015b270 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.177+0000 7f8da4bfb640 1 -- 172.21.15.49:0/572814792 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d980078e0 con 0x7f8da015b270 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.177+0000 7f8da4bfb640 1 -- 172.21.15.49:0/572814792 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98007cb0 con 0x7f8da015b270 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da015b270 msgr2=0x7f8da00c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da015b270 0x7f8da00c9da0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009dc0 tx=0x7f8d9802a5f0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 shutdown_connections 2024-05-09T08:57:38.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da015b270 0x7f8da00c9da0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 --2- 172.21.15.49:0/572814792 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da01552a0 0x7f8da015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 >> 172.21.15.49:0/572814792 conn(0x7f8da006b200 msgr2=0x7f8da006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 shutdown_connections 2024-05-09T08:57:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f8da7e61640 1 -- 172.21.15.49:0/572814792 wait complete. 2024-05-09T08:57:38.179 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:38.178+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:38.179 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.180+0000 7f8da7e61640 1 Processor -- start 2024-05-09T08:57:38.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.180+0000 7f8da7e61640 1 -- start start 2024-05-09T08:57:38.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da7e61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da0154ec0 0x7f8da00c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da7e61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da01552a0 0x7f8da014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da7e61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8da015b270 0x7f8da01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da5bfd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da0154ec0 0x7f8da00c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da5bfd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da0154ec0 0x7f8da00c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36686/0 (socket says 172.21.15.49:36686) 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.181+0000 7f8da53fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da01552a0 0x7f8da014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da53fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da01552a0 0x7f8da014e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:53098/0 (socket says 172.21.15.49:53098) 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 -- 172.21.15.49:0/1908611383 learned_addr learned my addr 172.21.15.49:0/1908611383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8da015e990 con 0x7f8da015b270 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8da015e6b0 con 0x7f8da01552a0 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8da00c76b0 con 0x7f8da0154ec0 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8da015b270 0x7f8da01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 -- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da01552a0 msgr2=0x7f8da014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8da01552a0 0x7f8da014e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 -- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8da015b270 msgr2=0x7f8da01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8da015b270 0x7f8da01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da0155a70 con 0x7f8da0154ec0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8da015b270 0x7f8da01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8da0154ec0 0x7f8da00c5b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c001eb0 tx=0x7f8d9c00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.182+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c017070 con 0x7f8da0154ec0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.183+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d9c005ce0 con 0x7f8da0154ec0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.183+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da0155d50 con 0x7f8da0154ec0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.183+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c007270 con 0x7f8da0154ec0 2024-05-09T08:57:38.184 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.183+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da0157ce0 con 0x7f8da0154ec0 2024-05-09T08:57:38.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.185+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8da004f660 con 0x7f8da0154ec0 2024-05-09T08:57:38.185 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.186+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d9c01a070 con 0x7f8da0154ec0 2024-05-09T08:57:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.186+0000 7f8d967fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d8406be30 0x7f8d8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.186+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5857+0+0 (secure 0 0 0) 0x7f8d9c08a260 con 0x7f8da0154ec0 2024-05-09T08:57:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d8406be30 0x7f8d8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v48) ==== 54+0+16 (secure 0 0 0) 0x7f8d9c08c8b0 con 0x7f8da0154ec0 2024-05-09T08:57:38.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f8da0157f20 0x7f8da0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8da0072600 con 0x7f8da0157f20 2024-05-09T08:57:38.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d8406be30 0x7f8d8406e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009560 tx=0x7f8d90009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.187+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f8da0157f20 0x7f8da0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.188+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f8da0157f20 0x7f8da0159ef0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.189+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da0072600 con 0x7f8da0157f20 2024-05-09T08:57:38.203 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:38.203 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:38.203 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:38.203 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:38.203 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:38.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:38.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.219+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da00c8fd0 con 0x7f8da0157f20 2024-05-09T08:57:38.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.222+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da00c8fd0 con 0x7f8da0157f20 2024-05-09T08:57:38.221 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.222+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f8da014d120 0x7f8da0148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.222+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8da01556a0 con 0x7f8da014d120 2024-05-09T08:57:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.222+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f8da014d120 0x7f8da0148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.223+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f8da014d120 0x7f8da0148280 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.224+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da01556a0 con 0x7f8da014d120 2024-05-09T08:57:38.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.255+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da00c6040 con 0x7f8da014d120 2024-05-09T08:57:38.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.257+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da00c6040 con 0x7f8da014d120 2024-05-09T08:57:38.256 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:38.257 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.257+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=49}) -- 0x7f8da0153c20 con 0x7f8da0154ec0 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.257+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8da014eeb0 con 0x7f8da0154ec0 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.258+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8d9c091070 con 0x7f8da0154ec0 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.258+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f8da0061c70 0x7f8da0062080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.258+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f8da0149090 con 0x7f8da0061c70 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.258+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f8da0061c70 0x7f8da0062080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.259+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f8da0061c70 0x7f8da0062080 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.260+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da0149090 con 0x7f8da0061c70 2024-05-09T08:57:38.278 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:38.279+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:38.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.289+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da0062810 con 0x7f8da0061c70 2024-05-09T08:57:38.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.292+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da0062810 con 0x7f8da0061c70 2024-05-09T08:57:38.291 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.292+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f8da00c4c20 0x7f8da00c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.292+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f8da00c5560 con 0x7f8da00c4c20 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.292+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f8da00c4c20 0x7f8da00c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.293+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f8da00c4c20 0x7f8da00c5020 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.293+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da00c5560 con 0x7f8da00c4c20 2024-05-09T08:57:38.315 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:38.315 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:38.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:38.315 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.635456101000003, 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:38.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:38.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:38.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:38.321 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.632036338999995, 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:38.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.631809829999995, 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.326 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.631756537000001, 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.334 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.583432966000004, 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:38.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.582907090999996, 2024-05-09T08:57:38.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:38.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.582659758999995, 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:38.343 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:38.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.347 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.577220556, 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.569261392000001, 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.568697212999993, 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:38.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.568610140000004, 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.361 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.568541081999996, 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:38.373 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.567182772999999, 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:38.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:38.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.378 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.567137525999996, 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:38.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:38.383 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.566882258999996, 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:38.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:38.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:38.388 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 74.566843563000006, 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:38.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.553900505000001, 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.393 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:38.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.397 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.553261556999999, 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:38.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.401 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.552707612999995, 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:38.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:38.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:38.406 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 68.549853475000006, 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:38.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:38.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:38.410 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.323+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da002c310 con 0x7f8da00c4c20 2024-05-09T08:57:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.325+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da002c310 con 0x7f8da00c4c20 2024-05-09T08:57:38.412 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:38.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:38.413 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:38.412+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.325+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f8da002ca30 0x7f8da002ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.325+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f8da00c5560 con 0x7f8da002ca30 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.325+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f8da002ca30 0x7f8da002ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.326+0000 7f8da5bfd640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f8da002ca30 0x7f8da002ce70 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.326+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da00c5560 con 0x7f8da002ca30 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.356+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da00c5560 con 0x7f8da002ca30 2024-05-09T08:57:38.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.358+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da00c5560 con 0x7f8da002ca30 2024-05-09T08:57:38.415 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.358+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f8da0209ba0 0x7f8da020bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.358+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f8da00c5560 con 0x7f8da0209ba0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.358+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f8da0209ba0 0x7f8da020bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.359+0000 7f8da53fc640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f8da0209ba0 0x7f8da020bf90 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.359+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da00c5560 con 0x7f8da0209ba0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.388+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da00c5560 con 0x7f8da0209ba0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.390+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da00c5560 con 0x7f8da0209ba0 2024-05-09T08:57:38.416 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.391+0000 7f8da7e61640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f8da020c9e0 0x7f8da020edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.391+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f8da002d3b0 con 0x7f8da020c9e0 2024-05-09T08:57:38.417 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.391+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f8da020c9e0 0x7f8da020edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.391+0000 7f8da63fe640 1 --2- 172.21.15.49:0/1908611383 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f8da020c9e0 0x7f8da020edd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.392+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8da002d3b0 con 0x7f8da020c9e0 2024-05-09T08:57:38.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.421+0000 7f8da7e61640 1 -- 172.21.15.49:0/1908611383 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8da002d3b0 con 0x7f8da020c9e0 2024-05-09T08:57:38.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.423+0000 7f8d967fc640 1 -- 172.21.15.49:0/1908611383 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8da002d3b0 con 0x7f8da020c9e0 2024-05-09T08:57:38.422 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:38.423 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:38.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:38.428 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:38.513 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:38.514+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:38.555 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:38.555 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:38.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:38.613 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:38.614+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:38.668 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:38.668 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:38.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:38.683 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:57:38.727 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:38.728+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:38.816 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:38.817+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:38.822 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:38.822 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:38.831 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:38.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.847+0000 7f4970464640 1 Processor -- start 2024-05-09T08:57:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.847+0000 7f4970464640 1 -- start start 2024-05-09T08:57:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f4970464640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f4970464640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f49680c7d10 0x7f49681634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f4970464640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4968070990 con 0x7f49680c9500 2024-05-09T08:57:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f4970464640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f49680ca5f0 con 0x7f49680c7d10 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f4970464640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f496815ccd0 con 0x7f4968152f30 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f496e200640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f496e200640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36702/0 (socket says 172.21.15.49:36702) 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f496e200640 1 -- 172.21.15.49:0/45392941 learned_addr learned my addr 172.21.15.49:0/45392941 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.849+0000 7f496ea01640 1 --2- 172.21.15.49:0/45392941 >> v2:172.21.15.183:3300/0 conn(0x7f49680c7d10 0x7f49681634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.850+0000 7f496e200640 1 -- 172.21.15.49:0/45392941 >> v2:172.21.15.183:3300/0 conn(0x7f49680c7d10 msgr2=0x7f49681634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.850+0000 7f496e200640 1 --2- 172.21.15.49:0/45392941 >> v2:172.21.15.183:3300/0 conn(0x7f49680c7d10 0x7f49681634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.850+0000 7f496e200640 1 -- 172.21.15.49:0/45392941 >> v1:172.21.15.183:6789/0 conn(0x7f49680c9500 legacy=0x7f49680c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.850+0000 7f496e200640 1 -- 172.21.15.49:0/45392941 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496815cfb0 con 0x7f4968152f30 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.851+0000 7f496e200640 1 --2- 172.21.15.49:0/45392941 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49640098a0 tx=0x7f496402a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835108e06cf982fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.851+0000 7f496d1fe640 1 -- 172.21.15.49:0/45392941 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964037070 con 0x7f4968152f30 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.851+0000 7f496d1fe640 1 -- 172.21.15.49:0/45392941 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4964032640 con 0x7f4968152f30 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.851+0000 7f496d1fe640 1 -- 172.21.15.49:0/45392941 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964032a10 con 0x7f4968152f30 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 -- 172.21.15.49:0/45392941 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 msgr2=0x7f49680c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 --2- 172.21.15.49:0/45392941 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49640098a0 tx=0x7f496402a330 comp rx=0 tx=0).stop 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 -- 172.21.15.49:0/45392941 shutdown_connections 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 --2- 172.21.15.49:0/45392941 >> v2:172.21.15.183:3300/0 conn(0x7f49680c7d10 0x7f49681634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 --2- 172.21.15.49:0/45392941 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4968152f30 0x7f49680c5890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 -- 172.21.15.49:0/45392941 >> 172.21.15.49:0/45392941 conn(0x7f49680ce390 msgr2=0x7f49680ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 -- 172.21.15.49:0/45392941 shutdown_connections 2024-05-09T08:57:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.852+0000 7f4970464640 1 -- 172.21.15.49:0/45392941 wait complete. 2024-05-09T08:57:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.854+0000 7f4970464640 1 Processor -- start 2024-05-09T08:57:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.854+0000 7f4970464640 1 -- start start 2024-05-09T08:57:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.855+0000 7f4970464640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f4970464640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49680c9500 0x7f49681f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:36710/0 (socket says 172.21.15.49:36710) 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 -- 172.21.15.49:0/640969098 learned_addr learned my addr 172.21.15.49:0/640969098 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496d9ff640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49680c9500 0x7f49681f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4968152f30 0x7f49681f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49680caa80 con 0x7f49680c9500 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49680c5dd0 con 0x7f4968152f30 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f49681f44b0 con 0x7f49680c7d10 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 -- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4968152f30 msgr2=0x7f49681f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4968152f30 0x7f49681f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 -- 172.21.15.49:0/640969098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49680c9500 msgr2=0x7f49681f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49680c9500 0x7f49681f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.856+0000 7f496e200640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681ee950 con 0x7f49680c7d10 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.857+0000 7f496e200640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f49640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.857+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964037040 con 0x7f49680c7d10 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.857+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4964042070 con 0x7f49680c7d10 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.857+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964030990 con 0x7f49680c7d10 2024-05-09T08:57:38.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.859+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496814f080 con 0x7f49680c7d10 2024-05-09T08:57:38.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.859+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681f4700 con 0x7f49680c7d10 2024-05-09T08:57:38.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.860+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496806f420 con 0x7f49680c7d10 2024-05-09T08:57:38.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.861+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4964030b30 con 0x7f49680c7d10 2024-05-09T08:57:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.861+0000 7f495effd640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 0x7f494c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.862+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5857+0+0 (secure 0 0 0) 0x7f49640ab690 con 0x7f49680c7d10 2024-05-09T08:57:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.862+0000 7f496d9ff640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 0x7f494c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:38.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.862+0000 7f496d9ff640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 0x7f494c06e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4968156400 tx=0x7f4958009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:38.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:38.866+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f496806f420 con 0x7f49680c7d10 2024-05-09T08:57:38.917 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:38.918+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:38.934 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:38.934 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:38.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:39.018 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:39.019+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:39.085 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.086 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.094 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:39.119 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:39.120+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:39.196 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.196 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:39.249 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:39.250+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:39.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.256+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f496804f660 con 0x7f49680c7d10 2024-05-09T08:57:39.308 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.308 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.318 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:39.359 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:39.360+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:39.421 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:39.422+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:39.461 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.461 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.470 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:39.554 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:39.555+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:39.613 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.613 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.621 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:39.663 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:39.664+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:39.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.679+0000 7f495effd640 1 -- 172.21.15.49:0/640969098 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v49) ==== 101+0+0 (secure 0 0 0) 0x7f4964030e20 con 0x7f49680c7d10 2024-05-09T08:57:39.681 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:57:39.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 msgr2=0x7f494c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:39.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 0x7f494c06e320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4968156400 tx=0x7f4958009040 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 msgr2=0x7f49680c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4964000c00 tx=0x7f49640046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 shutdown_connections 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f494c06be60 0x7f494c06e320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4968152f30 0x7f49681f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49680c9500 0x7f49681f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 --2- 172.21.15.49:0/640969098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49680c7d10 0x7f49680c51e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 >> 172.21.15.49:0/640969098 conn(0x7f49680ce390 msgr2=0x7f49681512b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 shutdown_connections 2024-05-09T08:57:39.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:39.685+0000 7f4970464640 1 -- 172.21.15.49:0/640969098 wait complete. 2024-05-09T08:57:39.724 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:39.725+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:39.789 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.789 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:39.842 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:39.843+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:39.909 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:39.910 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:39.920 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:39.925 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:39.926+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:40.026 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:40.027+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:40.073 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:40.073 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:40.083 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:40.127 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:40.128+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:40.201 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:40.201 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:40.211 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:40.228 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:40.229+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:40.328 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:40.329+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:40.426 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:40.427 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:40.428 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:40.429 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:40.430+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:40.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:40.530 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:40.531+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:40.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.586+0000 7f7fa0109640 1 Processor -- start 2024-05-09T08:57:40.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.587+0000 7f7fa0109640 1 -- start start 2024-05-09T08:57:40.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7fa0109640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 0x7f7f980cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7fa0109640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7f9d6a4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7f9d6a4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53888/0 (socket says 172.21.15.49:53888) 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7f9d6a4640 1 -- 172.21.15.49:0/1165670399 learned_addr learned my addr 172.21.15.49:0/1165670399 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f98072600 con 0x7f7f980c6640 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7f980728e0 con 0x7f7f98154f70 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f980c5880 con 0x7f7f98154b90 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.588+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/1165670399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 0x7f7f980cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.589+0000 7f7f9dea5640 1 -- 172.21.15.49:0/1165670399 >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 msgr2=0x7f7f980cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.589+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/1165670399 >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.589+0000 7f7f9dea5640 1 -- 172.21.15.49:0/1165670399 >> v1:172.21.15.183:6789/0 conn(0x7f7f980c6640 legacy=0x7f7f980c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.589+0000 7f7f9dea5640 1 -- 172.21.15.49:0/1165670399 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f980c5b60 con 0x7f7f98154b90 2024-05-09T08:57:40.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.589+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/1165670399 >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.590+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/1165670399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 0x7f7f980cb280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a80d0dd74201224 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.590+0000 7f7f9cea3640 1 -- 172.21.15.49:0/1165670399 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94037070 con 0x7f7f98154b90 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.590+0000 7f7f9cea3640 1 -- 172.21.15.49:0/1165670399 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f94032440 con 0x7f7f98154b90 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.590+0000 7f7f9cea3640 1 -- 172.21.15.49:0/1165670399 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94032810 con 0x7f7f98154b90 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.591+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 msgr2=0x7f7f980cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.591+0000 7f7fa0109640 1 --2- 172.21.15.49:0/1165670399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 0x7f7f980cb280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097a0 tx=0x7f7f9402a230 comp rx=0 tx=0).stop 2024-05-09T08:57:40.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.591+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 shutdown_connections 2024-05-09T08:57:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.592+0000 7f7fa0109640 1 --2- 172.21.15.49:0/1165670399 >> v2:172.21.15.49:3300/0 conn(0x7f7f98154f70 0x7f7f980cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.592+0000 7f7fa0109640 1 --2- 172.21.15.49:0/1165670399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f98154b90 0x7f7f980cb280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.592+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 >> 172.21.15.49:0/1165670399 conn(0x7f7f98074840 msgr2=0x7f7f98074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.592+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 shutdown_connections 2024-05-09T08:57:40.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.592+0000 7f7fa0109640 1 -- 172.21.15.49:0/1165670399 wait complete. 2024-05-09T08:57:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.594+0000 7f7fa0109640 1 Processor -- start 2024-05-09T08:57:40.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.594+0000 7f7fa0109640 1 -- start start 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f980c6640 0x7f7f981f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98154b90 0x7f7f98146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f98154f70 0x7f7f980b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9815af40 con 0x7f7f98154b90 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f980c6120 con 0x7f7f98154f70 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7f9dea5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f980c6640 0x7f7f981f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7f9e6a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f98154f70 0x7f7f980b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7f9e6a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f98154f70 0x7f7f980b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:53114/0 (socket says 172.21.15.49:53114) 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7f9e6a6640 1 -- 172.21.15.49:0/4038769608 learned_addr learned my addr 172.21.15.49:0/4038769608 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.595+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7f98071820 con 0x7f7f980c6640 2024-05-09T08:57:40.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98154b90 0x7f7f98146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 -- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f980c6640 msgr2=0x7f7f981f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7f980c6640 0x7f7f981f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 -- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98154b90 msgr2=0x7f7f98146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98154b90 0x7f7f98146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f981f0cf0 con 0x7f7f98154f70 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7f98154b90 0x7f7f98146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f98154f70 0x7f7f980b5e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c740 tx=0x7f7f9000cc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90016020 con 0x7f7f98154f70 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9000ce50 con 0x7f7f98154f70 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.596+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f98155690 con 0x7f7f98154f70 2024-05-09T08:57:40.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.597+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003970 con 0x7f7f98154f70 2024-05-09T08:57:40.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.597+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f981f0f90 con 0x7f7f98154f70 2024-05-09T08:57:40.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.598+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7f9804f660 con 0x7f7f98154f70 2024-05-09T08:57:40.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.599+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f90003b10 con 0x7f7f98154f70 2024-05-09T08:57:40.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.600+0000 7f7f8e7fc640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f7c06be30 0x7f7f7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.600+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5857+0+0 (secure 0 0 0) 0x7f7f9008ab00 con 0x7f7f98154f70 2024-05-09T08:57:40.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.600+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f7c06be30 0x7f7f7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.600+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7f7f90020020 con 0x7f7f98154f70 2024-05-09T08:57:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.601+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7f981f1190 0x7f7f9814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.601+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f98158920 con 0x7f7f981f1190 2024-05-09T08:57:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.601+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7f7c06be30 0x7f7f7c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f94034000 tx=0x7f7f9402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.601+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7f981f1190 0x7f7f9814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.601+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7f981f1190 0x7f7f9814a4d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.603+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f98158920 con 0x7f7f981f1190 2024-05-09T08:57:40.629 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:40.629 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:40.631 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:40.632+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:40.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.633+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f980cbde0 con 0x7f7f981f1190 2024-05-09T08:57:40.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.635+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f980cbde0 con 0x7f7f981f1190 2024-05-09T08:57:40.635 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.636+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7f981452a0 0x7f7f98147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.636+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7f980b52c0 con 0x7f7f981452a0 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.636+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7f981452a0 0x7f7f98147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.637+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7f981452a0 0x7f7f98147af0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.638+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f980b52c0 con 0x7f7f981452a0 2024-05-09T08:57:40.639 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:40.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.668+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f980cd490 con 0x7f7f981452a0 2024-05-09T08:57:40.670 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.670+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f980cd490 con 0x7f7f981452a0 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.670+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7f7f980cd1b0 con 0x7f7f98154f70 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.670+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f98147490 con 0x7f7f98154f70 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.671+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7f90054f90 con 0x7f7f98154f70 2024-05-09T08:57:40.671 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:40.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.671+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7f98148540 0x7f7f98061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.671+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f7f980b5950 con 0x7f7f98148540 2024-05-09T08:57:40.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.671+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7f98148540 0x7f7f98061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.672+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7f98148540 0x7f7f98061c70 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.673+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f980b5950 con 0x7f7f98148540 2024-05-09T08:57:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.703+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f98062400 con 0x7f7f98148540 2024-05-09T08:57:40.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.705+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f98062400 con 0x7f7f98148540 2024-05-09T08:57:40.705 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:40.705 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:40.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.706+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7f980629d0 0x7f7f98146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.706+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f7f98062400 con 0x7f7f980629d0 2024-05-09T08:57:40.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.706+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7f980629d0 0x7f7f98146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.707+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7f980629d0 0x7f7f98146070 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.707+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f98062400 con 0x7f7f980629d0 2024-05-09T08:57:40.732 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:40.733+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:40.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.736+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f98062400 con 0x7f7f980629d0 2024-05-09T08:57:40.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.738+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f98062400 con 0x7f7f980629d0 2024-05-09T08:57:40.737 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:40.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:40.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.738+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7f981468e0 0x7f7f9802c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.738+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f7f9802ca90 con 0x7f7f981468e0 2024-05-09T08:57:40.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.739+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7f981468e0 0x7f7f9802c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.739+0000 7f7f9dea5640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7f981468e0 0x7f7f9802c550 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.740+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9802ca90 con 0x7f7f981468e0 2024-05-09T08:57:40.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:57:40.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:57:40.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.770+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f9802ca90 con 0x7f7f981468e0 2024-05-09T08:57:40.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.773+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9802ca90 con 0x7f7f981468e0 2024-05-09T08:57:40.772 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.774+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7f9802d110 0x7f7f98207470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.774+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f7f9820a240 con 0x7f7f9802d110 2024-05-09T08:57:40.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.774+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7f9802d110 0x7f7f98207470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.774 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.775+0000 7f7f9d6a4640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7f9802d110 0x7f7f98207470 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.775 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.776+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9820a240 con 0x7f7f9802d110 2024-05-09T08:57:40.790 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:40.790 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:40.790 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:40.790 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:40.803 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.803+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f98207a30 con 0x7f7f9802d110 2024-05-09T08:57:40.804 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:40.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.805+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f98207a30 con 0x7f7f9802d110 2024-05-09T08:57:40.805 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.806+0000 7f7fa0109640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7f98208030 0x7f7f9820a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.806+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f7f9820d4e0 con 0x7f7f98208030 2024-05-09T08:57:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.806+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7f98208030 0x7f7f9820a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:40.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.806+0000 7f7f9e6a6640 1 --2- 172.21.15.49:0/4038769608 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7f98208030 0x7f7f9820a440 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:40.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.807+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9820d4e0 con 0x7f7f98208030 2024-05-09T08:57:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.822+0000 7f7fa0109640 1 -- 172.21.15.49:0/4038769608 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7f9820c200 con 0x7f7f98208030 2024-05-09T08:57:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:40.825+0000 7f7f8e7fc640 1 -- 172.21.15.49:0/4038769608 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9820c200 con 0x7f7f98208030 2024-05-09T08:57:40.824 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:40.829 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:40.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:40.837 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:40.838+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:40.920 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:40.921 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:40.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:40.933 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:40.934+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:41.034 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:41.035+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:41.044 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.363456205999995, 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.363436234999995, 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.362607468999997, 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.361604576999994, 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.361301745999995, 2024-05-09T08:57:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.320115188000003, 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.320103161999995, 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.320089691999996, 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.319065289999998, 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.318638000999997, 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.318521661000005, 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.317247664999996, 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.315498265000002, 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.315170120999994, 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.305041493000004, 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.314298492000006, 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.314289106000004, 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.314278062, 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.312543066000003, 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 71.282638860000006, 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:41.143 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:41.144 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:41.145 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:41.145+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:41.244 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:41.245+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:41.259 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:41.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:41.354 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:41.355+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:41.413 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:41.413 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:41.413 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:41.413 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:41.421 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:41.446 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:41.448+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:41.548 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:41.549+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:41.564 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.884534039000002, 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.881114276999995, 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.880887767999994, 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.880834475, 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.832510904000003, 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.831985028999995, 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.831737696999994, 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 83.826298494, 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:41.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:41.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:41.611 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.818339330000001, 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:41.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:41.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:41.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:41.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.817775151000006, 2024-05-09T08:57:41.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:41.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:41.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:41.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.817688078000003, 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:41.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:41.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:41.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:41.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.817619019999995, 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:41.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:41.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.816260710999998, 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.634 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:41.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.638 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.816215463999995, 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:41.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:41.643 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.815960196999995, 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:41.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:41.648 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.815921501000005, 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:41.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.802978443000001, 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:41.653 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:41.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:41.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.802339494999998, 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.801785550999995, 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 71.798931413000005, 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:41.679 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:41.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:41.681 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:41.681+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:41.702 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:57:41.781 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:41.783+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:41.784 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:41.785 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:41.785 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:41.785 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:41.794 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:41.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.812+0000 7f1ddceda640 1 Processor -- start 2024-05-09T08:57:41.811 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.812+0000 7f1ddceda640 1 -- start start 2024-05-09T08:57:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1ddceda640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1dd8154b90 0x7f1dd80cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1ddceda640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd8072600 con 0x7f1dd80c6d50 2024-05-09T08:57:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1ddceda640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dd80728e0 con 0x7f1dd8154f70 2024-05-09T08:57:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1ddceda640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd80c5880 con 0x7f1dd8154b90 2024-05-09T08:57:41.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1dd659c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1dd8154b90 0x7f1dd80cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1dd5d9b640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f1dd8154f70 0x7f1dd80c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:35944/0 (socket says 172.21.15.49:35944) 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.814+0000 7f1dd5d9b640 1 -- 172.21.15.49:0/1130991609 learned_addr learned my addr 172.21.15.49:0/1130991609 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.815+0000 7f1dd659c640 1 -- 172.21.15.49:0/1130991609 >> v1:172.21.15.183:6789/0 conn(0x7f1dd8154f70 legacy=0x7f1dd80c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.815+0000 7f1dd659c640 1 -- 172.21.15.49:0/1130991609 >> v1:172.21.15.49:6789/0 conn(0x7f1dd80c6d50 legacy=0x7f1dd8071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.815+0000 7f1dd559a640 1 -- 172.21.15.49:0/1130991609 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3042482502 0 0) 0x7f1dd8072600 con 0x7f1dd80c6d50 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.815+0000 7f1dd659c640 1 -- 172.21.15.49:0/1130991609 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80c5b60 con 0x7f1dd8154b90 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.815+0000 7f1dd659c640 1 --2- 172.21.15.49:0/1130991609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd8154b90 0x7f1dd80cb280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a298af972245ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1dd559a640 1 -- 172.21.15.49:0/1130991609 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8037070 con 0x7f1dd8154b90 2024-05-09T08:57:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1dd559a640 1 -- 172.21.15.49:0/1130991609 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8032640 con 0x7f1dd8154b90 2024-05-09T08:57:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1dd559a640 1 -- 172.21.15.49:0/1130991609 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8032a10 con 0x7f1dd8154b90 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 -- 172.21.15.49:0/1130991609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd8154b90 msgr2=0x7f1dd80cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 --2- 172.21.15.49:0/1130991609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd8154b90 0x7f1dd80cb280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80097a0 tx=0x7f1dc802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 -- 172.21.15.49:0/1130991609 shutdown_connections 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 --2- 172.21.15.49:0/1130991609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd8154b90 0x7f1dd80cb280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 -- 172.21.15.49:0/1130991609 >> 172.21.15.49:0/1130991609 conn(0x7f1dd8074840 msgr2=0x7f1dd8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 -- 172.21.15.49:0/1130991609 shutdown_connections 2024-05-09T08:57:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.816+0000 7f1ddceda640 1 -- 172.21.15.49:0/1130991609 wait complete. 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.818+0000 7f1ddceda640 1 Processor -- start 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.818+0000 7f1ddceda640 1 -- start start 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd80c6d50 0x7f1dd80b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 0x7f1dd80b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd815af40 con 0x7f1dd8154f70 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd80c6120 con 0x7f1dd80c6d50 2024-05-09T08:57:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1ddceda640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd80cb7c0 con 0x7f1dd8154b90 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd6d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53894/0 (socket says 172.21.15.49:53894) 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd6d9d640 1 -- 172.21.15.49:0/3976083321 learned_addr learned my addr 172.21.15.49:0/3976083321 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 0x7f1dd80b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd659c640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd80c6d50 0x7f1dd80b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 -- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd80c6d50 msgr2=0x7f1dd80b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd80c6d50 0x7f1dd80b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 -- 172.21.15.49:0/3976083321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 msgr2=0x7f1dd80b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 -- 172.21.15.49:0/3976083321 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd80ce680 con 0x7f1dd8154b90 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd6d9d640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.819+0000 7f1dd5d9b640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 0x7f1dd80b5e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c470 tx=0x7f1dcc00c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.820+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc017070 con 0x7f1dd8154b90 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.820+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dcc005ce0 con 0x7f1dd8154b90 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.820+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd80cf350 con 0x7f1dd8154b90 2024-05-09T08:57:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.820+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0072d0 con 0x7f1dd8154b90 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.820+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd80cf910 con 0x7f1dd8154b90 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.821+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd80cd8f0 con 0x7f1dd8154b90 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.821+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1dcc01a070 con 0x7f1dd8154b90 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.822+0000 7f1dc6ffd640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 0x7f1db806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.822+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(49..49 src has 1..49) ==== 5857+0+0 (secure 0 0 0) 0x7f1dcc089fd0 con 0x7f1dd8154b90 2024-05-09T08:57:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.822+0000 7f1dd659c640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 0x7f1db806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:41.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.823+0000 7f1dd659c640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 0x7f1db806e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8034000 tx=0x7f1dc802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:41.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:41.827+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1dd80cd8f0 con 0x7f1dd8154b90 2024-05-09T08:57:41.882 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:41.883+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:41.937 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:41.938 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:41.938 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:41.938 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:41.938 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:41.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:41.984 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:41.985+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:42.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.020+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f1dd804f660 con 0x7f1dd8154b90 2024-05-09T08:57:42.087 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:42.088+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:42.090 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:42.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.365803944000007, 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:42.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.365783344999997, 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:42.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:42.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.365778473999995, 2024-05-09T08:57:42.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:42.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.106 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.362220175000004, 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:42.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:42.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:42.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.362211947000006, 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:42.112 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:42.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:42.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:42.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.362208138, 2024-05-09T08:57:42.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:42.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:42.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:42.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.359652109999999, 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.124 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:42.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:42.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.345151302999994, 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:42.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.135 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.345135831999997, 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:42.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:42.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.345131088000002, 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:42.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.345124670999994, 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:42.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:42.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329148451999998, 2024-05-09T08:57:42.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:42.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:42.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329135750000006, 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:42.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:42.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329127833000001, 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:42.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:42.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329125325000007, 2024-05-09T08:57:42.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:42.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:42.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329122861000002, 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.175 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:42.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:42.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329107566000005, 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:42.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:42.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329103244999999, 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.329100104999995, 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:42.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:42.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.326640304999998, 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:42.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:42.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.199 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:42.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:42.201 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:42.202+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:42.302 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:42.303+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:42.303 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:42.304 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:42.304 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:42.304 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:42.313 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:42.403 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:42.404+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:42.455 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:42.455 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:42.455 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:42.455 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:42.455 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:42.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:42.504 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:42.505+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:42.568 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:42.568 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:42.569 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.887104346000001, 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:42.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:42.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.887101478999995, 2024-05-09T08:57:42.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:42.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.887097834000002, 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:42.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:42.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.844288668999994, 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.590 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:42.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:42.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.839496679999996, 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.600 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:42.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:42.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:42.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.836990311999998, 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.611 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:42.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:42.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.835518922000006, 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:42.619 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:42.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:42.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:42.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.835443084000005, 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.628 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:42.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:42.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822621674000004, 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.633 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:42.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:42.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.638 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822600675999993, 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:42.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:42.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.644 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822596282000006, 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:42.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:42.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822591224000007, 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.650 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:42.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:42.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.655 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822587743, 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:42.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:42.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822583124999994, 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:42.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:42.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:42.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822579656000002, 2024-05-09T08:57:42.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:42.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:42.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:42.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822576354999995, 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:42.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:42.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:42.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 78.822263285999995, 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:42.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:42.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.806187464000004, 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.683 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:42.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.806182785000004, 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:42.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:42.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:42.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.694 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 72.806176766999997, 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:42.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:42.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:42.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:42.703 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:42.704 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:42.704 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:42.704 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-09T08:57:42.704 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-09T08:57:42.706 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:42.707 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:42.706+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:42.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.687+0000 7f1dc6ffd640 1 -- 172.21.15.49:0/3976083321 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v50) ==== 87+0+0 (secure 0 0 0) 0x7f1dcc058320 con 0x7f1dd8154b90 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 msgr2=0x7f1db806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 0x7f1db806e320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8034000 tx=0x7f1dc802e000 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 msgr2=0x7f1dd80b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 0x7f1dd80b5e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c470 tx=0x7f1dcc00c930 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 shutdown_connections 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1db806be60 0x7f1db806e320 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1dd8154f70 0x7f1dd80b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1dd8154b90 0x7f1dd80b5e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 --2- 172.21.15.49:0/3976083321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dd80c6d50 0x7f1dd80b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 >> 172.21.15.49:0/3976083321 conn(0x7f1dd8074840 msgr2=0x7f1dd814a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:42.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 shutdown_connections 2024-05-09T08:57:42.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.689+0000 7f1ddceda640 1 -- 172.21.15.49:0/3976083321 wait complete. 2024-05-09T08:57:42.805 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:42.806+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:42.830 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:42.850 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:42.850 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:42.850 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:42.850 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:42.859 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.886+0000 7ff2ef4ba640 1 Processor -- start 2024-05-09T08:57:42.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.886+0000 7ff2ef4ba640 1 -- start start 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.887+0000 7ff2ef4ba640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff2e81517f0 0x7ff2e815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.887+0000 7ff2ef4ba640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff2e80cc190 0x7ff2e80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2ef4ba640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e80ccb20 con 0x7ff2e8075200 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2ef4ba640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e8151bd0 con 0x7ff2e80cc190 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2ef4ba640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e815e260 con 0x7ff2e81517f0 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff2e80cc190 0x7ff2e80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff2e80cc190 0x7ff2e80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41970/0 (socket says 172.21.15.49:41970) 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 -- 172.21.15.49:0/541327484 learned_addr learned my addr 172.21.15.49:0/541327484 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2ed256640 1 --2- 172.21.15.49:0/541327484 >> v2:172.21.15.183:3300/0 conn(0x7ff2e81517f0 0x7ff2e815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 -- 172.21.15.49:0/541327484 >> v2:172.21.15.183:3300/0 conn(0x7ff2e81517f0 msgr2=0x7ff2e815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:42.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 --2- 172.21.15.49:0/541327484 >> v2:172.21.15.183:3300/0 conn(0x7ff2e81517f0 0x7ff2e815b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 -- 172.21.15.49:0/541327484 >> v1:172.21.15.49:6789/0 conn(0x7ff2e8075200 legacy=0x7ff2e80ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2dffff640 1 -- 172.21.15.49:0/541327484 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763705463 0 0) 0x7ff2e80ccb20 con 0x7ff2e8075200 2024-05-09T08:57:42.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.888+0000 7ff2eda57640 1 -- 172.21.15.49:0/541327484 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e815e520 con 0x7ff2e80cc190 2024-05-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.889+0000 7ff2eda57640 1 --2- 172.21.15.49:0/541327484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80cc5e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80c78a0 tx=0x7ff2e0032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef2d90417411ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.889+0000 7ff2dffff640 1 -- 172.21.15.49:0/541327484 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0033b10 con 0x7ff2e80cc190 2024-05-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.889+0000 7ff2dffff640 1 -- 172.21.15.49:0/541327484 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e0033cb0 con 0x7ff2e80cc190 2024-05-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.889+0000 7ff2dffff640 1 -- 172.21.15.49:0/541327484 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0033e60 con 0x7ff2e80cc190 2024-05-09T08:57:42.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/541327484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 msgr2=0x7ff2e80cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 --2- 172.21.15.49:0/541327484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80cc5e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80c78a0 tx=0x7ff2e0032fd0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/541327484 shutdown_connections 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 --2- 172.21.15.49:0/541327484 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80cc5e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 --2- 172.21.15.49:0/541327484 >> v2:172.21.15.183:3300/0 conn(0x7ff2e81517f0 0x7ff2e815b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.890+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/541327484 >> 172.21.15.49:0/541327484 conn(0x7ff2e806bd00 msgr2=0x7ff2e8074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.891+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/541327484 shutdown_connections 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.891+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/541327484 wait complete. 2024-05-09T08:57:42.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.892+0000 7ff2ef4ba640 1 Processor -- start 2024-05-09T08:57:42.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.892+0000 7ff2ef4ba640 1 -- start start 2024-05-09T08:57:42.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e8075200 0x7ff2e815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e81517f0 0x7ff2e80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e815edc0 con 0x7ff2e80cc190 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e815eae0 con 0x7ff2e81517f0 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ef4ba640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e815f0a0 con 0x7ff2e8075200 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2ed256640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e8075200 0x7ff2e815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41976/0 (socket says 172.21.15.49:41976) 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 -- 172.21.15.49:0/392603145 learned_addr learned my addr 172.21.15.49:0/392603145 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eda57640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e81517f0 0x7ff2e80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 -- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e8075200 msgr2=0x7ff2e815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e8075200 0x7ff2e815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 -- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e81517f0 msgr2=0x7ff2e80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e81517f0 0x7ff2e80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.894+0000 7ff2eca55640 1 -- 172.21.15.49:0/392603145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e80ca550 con 0x7ff2e80cc190 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2eca55640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c460 tx=0x7ff2d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8005480 con 0x7ff2e80cc190 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/392603145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e80ca7f0 con 0x7ff2e80cc190 2024-05-09T08:57:42.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d80035f0 con 0x7ff2e80cc190 2024-05-09T08:57:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8003a00 con 0x7ff2e80cc190 2024-05-09T08:57:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.895+0000 7ff2ef4ba640 1 -- 172.21.15.49:0/392603145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e814a120 con 0x7ff2e80cc190 2024-05-09T08:57:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.897+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e8068560 con 0x7ff2e80cc190 2024-05-09T08:57:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.897+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2d8003ba0 con 0x7ff2e80cc190 2024-05-09T08:57:42.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.898+0000 7ff2ddffb640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 0x7ff2cc06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.898+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5857+0+0 (secure 0 0 0) 0x7ff2d808a9e0 con 0x7ff2e80cc190 2024-05-09T08:57:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.899+0000 7ff2ed256640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 0x7ff2cc06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.899+0000 7ff2ed256640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 0x7ff2cc06e4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009aa0 tx=0x7ff2e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.907+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff2d8058cb0 con 0x7ff2e80cc190 2024-05-09T08:57:42.945 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:42.946+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:42.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.962+0000 7f7feeddb640 1 Processor -- start 2024-05-09T08:57:42.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.962+0000 7f7feeddb640 1 -- start start 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7feeddb640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fe80c6a40 0x7f7fe81631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7feeddb640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7fe815eec0 0x7f7fe815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7feeddb640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe8072950 con 0x7f7fe80c6e20 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7feeddb640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe8163730 con 0x7f7fe80c6a40 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7feeddb640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe80b65b0 con 0x7f7fe815eec0 2024-05-09T08:57:42.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fecb77640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7fe80c6a40 0x7f7fe81631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fe7fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7fe80c6e20 0x7f7fe815e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59614/0 (socket says 172.21.15.49:59614) 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fe7fff640 1 -- 172.21.15.49:0/3969854376 learned_addr learned my addr 172.21.15.49:0/3969854376 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fed378640 1 --2- 172.21.15.49:0/3969854376 >> v2:172.21.15.183:3300/0 conn(0x7f7fe815eec0 0x7f7fe815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fecb77640 1 -- 172.21.15.49:0/3969854376 >> v2:172.21.15.183:3300/0 conn(0x7f7fe815eec0 msgr2=0x7f7fe815f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fecb77640 1 --2- 172.21.15.49:0/3969854376 >> v2:172.21.15.183:3300/0 conn(0x7f7fe815eec0 0x7f7fe815f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fecb77640 1 -- 172.21.15.49:0/3969854376 >> v1:172.21.15.49:6789/0 conn(0x7f7fe80c6e20 legacy=0x7f7fe815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.964+0000 7f7fecb77640 1 -- 172.21.15.49:0/3969854376 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe8072bf0 con 0x7f7fe80c6a40 2024-05-09T08:57:42.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.966+0000 7f7fecb77640 1 --2- 172.21.15.49:0/3969854376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81631f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80b5ba0 tx=0x7f7fdc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be75b65f4a7025a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.966+0000 7f7fe77fe640 1 -- 172.21.15.49:0/3969854376 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc036070 con 0x7f7fe80c6a40 2024-05-09T08:57:42.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.966+0000 7f7fe77fe640 1 -- 172.21.15.49:0/3969854376 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fdc031600 con 0x7f7fe80c6a40 2024-05-09T08:57:42.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.966+0000 7f7fe77fe640 1 -- 172.21.15.49:0/3969854376 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0319d0 con 0x7f7fe80c6a40 2024-05-09T08:57:42.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 -- 172.21.15.49:0/3969854376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 msgr2=0x7f7fe81631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 --2- 172.21.15.49:0/3969854376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81631f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80b5ba0 tx=0x7f7fdc02a420 comp rx=0 tx=0).stop 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 -- 172.21.15.49:0/3969854376 shutdown_connections 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 --2- 172.21.15.49:0/3969854376 >> v2:172.21.15.183:3300/0 conn(0x7f7fe815eec0 0x7f7fe815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 --2- 172.21.15.49:0/3969854376 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81631f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 -- 172.21.15.49:0/3969854376 >> 172.21.15.49:0/3969854376 conn(0x7f7fe8074840 msgr2=0x7f7fe8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 -- 172.21.15.49:0/3969854376 shutdown_connections 2024-05-09T08:57:42.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.967+0000 7f7feeddb640 1 -- 172.21.15.49:0/3969854376 wait complete. 2024-05-09T08:57:42.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.969+0000 7f7feeddb640 1 Processor -- start 2024-05-09T08:57:42.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.969+0000 7f7feeddb640 1 -- start start 2024-05-09T08:57:42.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.970+0000 7f7feeddb640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81f8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7fecb77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81f8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7fecb77640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81f8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:41990/0 (socket says 172.21.15.49:41990) 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7fecb77640 1 -- 172.21.15.49:0/1717986929 learned_addr learned my addr 172.21.15.49:0/1717986929 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fe80c6e20 0x7f7fe815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7fe815eec0 0x7f7fe80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80c5e80 con 0x7f7fe80c6a40 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe80b62d0 con 0x7f7fe80c6e20 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fe80c6e20 0x7f7fe815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7fe815eec0 0x7f7fe80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.971+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe80c6000 con 0x7f7fe815eec0 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 -- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7fe815eec0 msgr2=0x7f7fe80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7fe815eec0 0x7f7fe80b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 -- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 msgr2=0x7f7fe81f8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7fe80c6a40 0x7f7fe81f8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe81f9080 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fe80c6e20 0x7f7fe815a950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4000f60 tx=0x7f7fd400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4017070 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4005ce0 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4007310 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe80c8560 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.972+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81606b0 con 0x7f7fe80c6e20 2024-05-09T08:57:42.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.973+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7fe804f660 con 0x7f7fe80c6e20 2024-05-09T08:57:42.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.975+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7fd401a070 con 0x7f7fe80c6e20 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7fe57fa640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7fcc06be30 0x7f7fcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5857+0+0 (secure 0 0 0) 0x7f7fd408a9e0 con 0x7f7fe80c6e20 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7fcc06be30 0x7f7fcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v50) ==== 54+0+16 (secure 0 0 0) 0x7f7fd4021070 con 0x7f7fe80c6e20 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7fe81610c0 0x7f7fe80cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.976+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fe81608b0 con 0x7f7fe81610c0 2024-05-09T08:57:42.976 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.977+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7fe81610c0 0x7f7fe80cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.977+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7fcc06be30 0x7f7fcc06e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009560 tx=0x7f7fdc008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.977+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7fe81610c0 0x7f7fe80cb6b0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.978+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe81608b0 con 0x7f7fe81610c0 2024-05-09T08:57:42.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.993+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe80cbbf0 con 0x7f7fe81610c0 2024-05-09T08:57:42.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.995+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe80cbbf0 con 0x7f7fe81610c0 2024-05-09T08:57:42.994 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:42.995 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:42.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.995+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7fe80b3a70 0x7f7fe80b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:42.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.995+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7fe80b4390 con 0x7f7fe80b3a70 2024-05-09T08:57:42.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.995+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7fe80b3a70 0x7f7fe80b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:42.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.996+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7fe80b3a70 0x7f7fe80b3e50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:42.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:42.996+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe80b4390 con 0x7f7fe80b3a70 2024-05-09T08:57:43.004 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:43.004 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:43.005 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:43.005 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:43.005 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:43.008 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:43.009+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:43.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.012+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe815ae90 con 0x7f7fe80b3a70 2024-05-09T08:57:43.013 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:43.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.013+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe815ae90 con 0x7f7fe80b3a70 2024-05-09T08:57:43.015 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.014+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f7fe81f8260 con 0x7f7fe80c6e20 2024-05-09T08:57:43.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.014+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7fe81566a0 con 0x7f7fe80c6e20 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.014+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7fd4054ea0 con 0x7f7fe80c6e20 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.014+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7fe81571e0 0x7f7fe8061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.015+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f7fe80621b0 con 0x7f7fe81571e0 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.015+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7fe81571e0 0x7f7fe8061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.015+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7fe81571e0 0x7f7fe8061c70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.017 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.016+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe80621b0 con 0x7f7fe81571e0 2024-05-09T08:57:43.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.031+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe80626c0 con 0x7f7fe81571e0 2024-05-09T08:57:43.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.033+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe80626c0 con 0x7f7fe81571e0 2024-05-09T08:57:43.033 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:43.033 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:43.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.033+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7fe8158ad0 0x7f7fe8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.034+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f7fe8159470 con 0x7f7fe8158ad0 2024-05-09T08:57:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.034+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7fe8158ad0 0x7f7fe8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.034+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7fe8158ad0 0x7f7fe8158f30 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.034 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.035+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe8159470 con 0x7f7fe8158ad0 2024-05-09T08:57:43.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.049+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe802c310 con 0x7f7fe8158ad0 2024-05-09T08:57:43.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.051+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe802c310 con 0x7f7fe8158ad0 2024-05-09T08:57:43.050 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:43.051 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:43.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.051+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7fe802c990 0x7f7fe802cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.051+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f7fe802d330 con 0x7f7fe802c990 2024-05-09T08:57:43.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.052+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7fe802c990 0x7f7fe802cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.052+0000 7f7fecb77640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7fe802c990 0x7f7fe802cdf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.053+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe802d330 con 0x7f7fe802c990 2024-05-09T08:57:43.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.067+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe802d330 con 0x7f7fe802c990 2024-05-09T08:57:43.068 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.069+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe802d330 con 0x7f7fe802c990 2024-05-09T08:57:43.068 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:43.069 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:43.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.069+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7fe8209cb0 0x7f7fe820c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.069+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f7fe802d4a0 con 0x7f7fe8209cb0 2024-05-09T08:57:43.069 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.069+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7fe8209cb0 0x7f7fe820c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.070+0000 7f7fe7fff640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7fe8209cb0 0x7f7fe820c0a0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.070 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.071+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe802d4a0 con 0x7f7fe8209cb0 2024-05-09T08:57:43.084 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.085+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe802d4a0 con 0x7f7fe8209cb0 2024-05-09T08:57:43.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.087+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe802d4a0 con 0x7f7fe8209cb0 2024-05-09T08:57:43.086 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.087+0000 7f7feeddb640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7fe820caf0 0x7f7fe820eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.087+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f7fe8211d30 con 0x7f7fe820caf0 2024-05-09T08:57:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.087+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7fe820caf0 0x7f7fe820eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.088+0000 7f7fed378640 1 --2- 172.21.15.49:0/1717986929 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7fe820caf0 0x7f7fe820eee0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.088+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe8211d30 con 0x7f7fe820caf0 2024-05-09T08:57:43.102 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.103+0000 7f7feeddb640 1 -- 172.21.15.49:0/1717986929 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7fe820f6f0 con 0x7f7fe820caf0 2024-05-09T08:57:43.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.105+0000 7f7fe57fa640 1 -- 172.21.15.49:0/1717986929 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7fe820f6f0 con 0x7f7fe820caf0 2024-05-09T08:57:43.106 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:43.107 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:43.108 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:43.109+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:43.108 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:43.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:43.157 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.158 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.166 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:43.210 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:43.211+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:43.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.282+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ff2e815ba40 con 0x7ff2e80cc190 2024-05-09T08:57:43.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.284+0000 7ff2ddffb640 1 -- 172.21.15.49:0/392603145 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+35167 (secure 0 0 0) 0x7ff2d805cfb0 con 0x7ff2e80cc190 2024-05-09T08:57:43.284 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:57:43.284 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 50, 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "aab8b2de-6010-439c-b8fd-8bc72445265c", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "created": "2024-05-09T08:55:44.445697+0000", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "modified": "2024-05-09T08:57:42.679911+0000", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "last_up_change": "2024-05-09T08:57:15.391418+0000", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "last_in_change": "2024-05-09T08:55:46.620125+0000", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_num": 5801984, 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_set": [ 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "noscrub", 2024-05-09T08:57:43.285 INFO:teuthology.orchestra.run.smithi049.stdout: "pglog_hardlimit", 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "purged_snapdirs", 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_deletes", 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "sortbitwise" 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_version": 4, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "full_ratio": 0.94999998807907104, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_snapshot": "", 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_max": 3, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd": 8, 2024-05-09T08:57:43.286 INFO:teuthology.orchestra.run.smithi049.stdout: "require_min_compat_client": "luminous", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "min_compat_client": "luminous", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "require_osd_release": "squid", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "allow_crimson": false, 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "pools": [ 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 1, 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": ".mgr", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:55:56.616182+0000", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:57:43.287 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 1, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 1, 2024-05-09T08:57:43.288 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 1, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 1, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 1, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "16", 2024-05-09T08:57:43.289 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:57:43.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:57:43.291 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:57:43.292 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_max": 32, 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 1 2024-05-09T08:57:43.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": {} 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 8, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 8, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 0.25, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 2, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 2, 2024-05-09T08:57:43.294 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 0.97271156311035156, 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 0.97000002861022949, 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 2, 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cephfs_metadata", 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:55:59.400583+0000", 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:57:43.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:57:43.296 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 32, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 32, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 32, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 32, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 32, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:57:43.297 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "19", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:57:43.298 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:57:43.299 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:57:43.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:57:43.301 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_bias": 4, 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 16, 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_priority": 5 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs": { 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: "metadata": "cephfs" 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 1.7400000095367432, 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 1.7400000095367432, 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 0.87999999523162842, 2024-05-09T08:57:43.303 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 1.5299999713897705, 2024-05-09T08:57:43.304 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 1.5299999713897705, 2024-05-09T08:57:43.304 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 0.97271156311035156, 2024-05-09T08:57:43.304 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 0.97000002861022949, 2024-05-09T08:57:43.304 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-05-09T08:57:43.304 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 3, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cephfs_data", 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-05-09T08:56:00.024628+0000", 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-05-09T08:57:43.305 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 64, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 64, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 64, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 64, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 64, 2024-05-09T08:57:43.306 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-05-09T08:57:43.307 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "20", 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-05-09T08:57:43.310 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-05-09T08:57:43.311 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.312 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-05-09T08:57:43.313 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 64, 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "target_size_ratio": 0.89999997615814209 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-05-09T08:57:43.314 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs": { 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "data": "cephfs" 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "read_balance": { 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "score_type": "Fair distribution", 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "score_acting": 1.3600000143051147, 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "score_stable": 1.3600000143051147, 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "optimal_score": 0.87999999523162842, 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_acting": 1.2000000476837158, 2024-05-09T08:57:43.315 INFO:teuthology.orchestra.run.smithi049.stdout: "raw_score_stable": 1.2000000476837158, 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity_weighted": 0.97271156311035156, 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity": 0.97000002861022949, 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-05-09T08:57:43.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "b59d26bf-6806-4e49-b117-9d83d3a24243", 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.317 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-05-09T08:57:43.318 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6802", 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6803", 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6806", 2024-05-09T08:57:43.320 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6807", 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.321 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6804", 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6805", 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1154213557 2024-05-09T08:57:43.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6801/1154213557", 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6803/1154213557", 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6807/1154213557", 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6805/1154213557", 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "d579c6a2-0370-4c21-96fb-6420583dbe74", 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.324 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6816", 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6817", 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.326 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6818", 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6819", 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.327 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6823", 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6825", 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6820", 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.329 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6821", 2024-05-09T08:57:43.330 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3346547076 2024-05-09T08:57:43.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.330 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.330 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6817/3346547076", 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6819/3346547076", 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6825/3346547076", 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6821/3346547076", 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.331 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "bb2ad0e5-2ec1-4647-bc42-3d20911e038a", 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 0, 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.332 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 0.7816925048828125, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 24, 2024-05-09T08:57:43.333 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 39, 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6808", 2024-05-09T08:57:43.334 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6809", 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.336 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.336 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.336 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6810", 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.337 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6811", 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.338 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6814", 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6815", 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.340 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6812", 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6813", 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 136930210 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6809/136930210", 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6811/136930210", 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6815/136930210", 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6813/136930210", 2024-05-09T08:57:43.343 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.344 INFO:teuthology.orchestra.run.smithi049.stdout: "exists" 2024-05-09T08:57:43.344 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.344 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-05-09T08:57:43.345 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "546a2935-7144-4c29-b647-e77d89072d60", 2024-05-09T08:57:43.345 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.345 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.345 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.345 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.346 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6822", 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6824", 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6826", 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6827", 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6830", 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6831", 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.349 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6828", 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6829", 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 1536339711 2024-05-09T08:57:43.350 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6824/1536339711", 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6827/1536339711", 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6831/1536339711", 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6829/1536339711", 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.351 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "f4bb828b-6036-4886-aec7-be53c00ac603", 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6808", 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6809", 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6810", 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6811", 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.354 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6814", 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6815", 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.355 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6812", 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6813", 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 173400960 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.356 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6809/173400960", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6811/173400960", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6815/173400960", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6813/173400960", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "41625c33-9f16-4279-b2aa-a63ac0aa6320", 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.357 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6800", 2024-05-09T08:57:43.358 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6801", 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.359 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6802", 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6803", 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.360 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6806", 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6807", 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6804", 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6805", 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 4279483611 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.362 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6801/4279483611", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6803/4279483611", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6807/4279483611", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6805/4279483611", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "9177552d-bc69-409e-9654-e030c3a1afba", 2024-05-09T08:57:43.363 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.364 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6824", 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6825", 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6826", 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6827", 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.366 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6830", 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6831", 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6828", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6829", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 3521264583 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6825/3521264583", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6827/3521264583", 2024-05-09T08:57:43.368 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6831/3521264583", 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6829/3521264583", 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "d3b1821c-1ed4-4286-b7de-030fda235fbf", 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-05-09T08:57:43.369 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 39, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6816", 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6817", 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6818", 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6819", 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6822", 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6823", 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6820", 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.183:6821", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 2743896476 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.183:6817/2743896476", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.183:6819/2743896476", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.183:6823/2743896476", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.183:6821/2743896476", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-05-09T08:57:43.374 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_xinfo": [ 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.763716+0000", 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.782386+0000", 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-05-09T08:57:43.376 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.712986+0000", 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 39 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.377 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.846199+0000", 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.735683+0000", 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.776861+0000", 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.379 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.767124+0000", 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540701547738038271, 2024-05-09T08:57:43.380 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-05-09T08:55:51.746641+0000", 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap": [], 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_items": [ 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "pgid": "3.1d", 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "mappings": [ 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "from": 5, 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: "to": 2 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.381 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "pgid": "3.24", 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "mappings": [ 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "from": 3, 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "to": 1 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "pgid": "3.39", 2024-05-09T08:57:43.382 INFO:teuthology.orchestra.run.smithi049.stdout: "mappings": [ 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "from": 7, 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "to": 2 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "pgid": "3.3c", 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "mappings": [ 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "from": 5, 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: "to": 2 2024-05-09T08:57:43.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_primaries": [], 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_temp": [], 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_temp": [], 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklist": {}, 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "range_blocklist": {}, 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profiles": { 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "default": { 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-failure-domain": "osd", 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "k": "2", 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "m": "1", 2024-05-09T08:57:43.384 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin": "jerasure", 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "technique": "reed_sol_van" 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps_queue": [], 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "new_removed_snaps": [], 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "new_purged_snaps": [], 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_node_flags": {}, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "device_class_flags": {}, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode": { 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_enabled": false, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_bucket_count": 0, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "degraded_stretch_mode": 0, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "recovering_stretch_mode": 0, 2024-05-09T08:57:43.385 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_bucket": 0 2024-05-09T08:57:43.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:43.386 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 msgr2=0x7ff2cc06e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 0x7ff2cc06e4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009aa0 tx=0x7ff2e4009190 comp rx=0 tx=0).stop 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 msgr2=0x7ff2e80b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c460 tx=0x7ff2d800c920 comp rx=0 tx=0).stop 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 shutdown_connections 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff2cc06c000 0x7ff2cc06e4c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2e81517f0 0x7ff2e80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2e80cc190 0x7ff2e80b4c10 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff2d800c460 tx=0x7ff2d800c920 comp rx=0 tx=0).stop 2024-05-09T08:57:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 --2- 172.21.15.49:0/392603145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2e8075200 0x7ff2e815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.287+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 >> 172.21.15.49:0/392603145 conn(0x7ff2e806bd00 msgr2=0x7ff2e814fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.288+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 shutdown_connections 2024-05-09T08:57:43.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.288+0000 7ff2d37fe640 1 -- 172.21.15.49:0/392603145 wait complete. 2024-05-09T08:57:43.391 INFO:tasks.thrashosds.thrasher:j is {'epoch': 50, 'fsid': 'aab8b2de-6010-439c-b8fd-8bc72445265c', 'created': '2024-05-09T08:55:44.445697+0000', 'modified': '2024-05-09T08:57:42.679911+0000', 'last_up_change': '2024-05-09T08:57:15.391418+0000', 'last_in_change': '2024-05-09T08:55:46.620125+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9727115631103516, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:55:59.400583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.9727115631103516, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:56:00.024628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.2000000476837158, 'raw_score_stable': 1.2000000476837158, 'primary_affinity_weighted': 0.9727115631103516, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}], 'osds': [{'osd': 0, 'uuid': 'b59d26bf-6806-4e49-b117-9d83d3a24243', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6800', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6801', 'nonce': 1154213557}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6802', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6803', 'nonce': 1154213557}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6806', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6807', 'nonce': 1154213557}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6804', 'nonce': 1154213557}, {'type': 'v1', 'addr': '172.21.15.49:6805', 'nonce': 1154213557}]}, 'public_addr': '172.21.15.49:6801/1154213557', 'cluster_addr': '172.21.15.49:6803/1154213557', 'heartbeat_back_addr': '172.21.15.49:6807/1154213557', 'heartbeat_front_addr': '172.21.15.49:6805/1154213557', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd579c6a2-0370-4c21-96fb-6420583dbe74', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6816', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6817', 'nonce': 3346547076}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6818', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6819', 'nonce': 3346547076}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6823', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6825', 'nonce': 3346547076}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6820', 'nonce': 3346547076}, {'type': 'v1', 'addr': '172.21.15.49:6821', 'nonce': 3346547076}]}, 'public_addr': '172.21.15.49:6817/3346547076', 'cluster_addr': '172.21.15.49:6819/3346547076', 'heartbeat_back_addr': '172.21.15.49:6825/3346547076', 'heartbeat_front_addr': '172.21.15.49:6821/3346547076', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'bb2ad0e5-2ec1-4647-bc42-3d20911e038a', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0.7816925048828125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 24, 'down_at': 39, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6808', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6809', 'nonce': 136930210}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6810', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6811', 'nonce': 136930210}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6814', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6815', 'nonce': 136930210}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6812', 'nonce': 136930210}, {'type': 'v1', 'addr': '172.21.15.49:6813', 'nonce': 136930210}]}, 'public_addr': '172.21.15.49:6809/136930210', 'cluster_addr': '172.21.15.49:6811/136930210', 'heartbeat_back_addr': '172.21.15.49:6815/136930210', 'heartbeat_front_addr': '172.21.15.49:6813/136930210', 'state': ['exists']}, {'osd': 3, 'uuid': '546a2935-7144-4c29-b647-e77d89072d60', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6822', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6824', 'nonce': 1536339711}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6826', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6827', 'nonce': 1536339711}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6830', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6831', 'nonce': 1536339711}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6828', 'nonce': 1536339711}, {'type': 'v1', 'addr': '172.21.15.49:6829', 'nonce': 1536339711}]}, 'public_addr': '172.21.15.49:6824/1536339711', 'cluster_addr': '172.21.15.49:6827/1536339711', 'heartbeat_back_addr': '172.21.15.49:6831/1536339711', 'heartbeat_front_addr': '172.21.15.49:6829/1536339711', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f4bb828b-6036-4886-aec7-be53c00ac603', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6809', 'nonce': 173400960}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6811', 'nonce': 173400960}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6815', 'nonce': 173400960}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 173400960}, {'type': 'v1', 'addr': '172.21.15.183:6813', 'nonce': 173400960}]}, 'public_addr': '172.21.15.183:6809/173400960', 'cluster_addr': '172.21.15.183:6811/173400960', 'heartbeat_back_addr': '172.21.15.183:6815/173400960', 'heartbeat_front_addr': '172.21.15.183:6813/173400960', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '41625c33-9f16-4279-b2aa-a63ac0aa6320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6801', 'nonce': 4279483611}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6803', 'nonce': 4279483611}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6807', 'nonce': 4279483611}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 4279483611}, {'type': 'v1', 'addr': '172.21.15.183:6805', 'nonce': 4279483611}]}, 'public_addr': '172.21.15.183:6801/4279483611', 'cluster_addr': '172.21.15.183:6803/4279483611', 'heartbeat_back_addr': '172.21.15.183:6807/4279483611', 'heartbeat_front_addr': '172.21.15.183:6805/4279483611', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9177552d-bc69-409e-9654-e030c3a1afba', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6824', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6825', 'nonce': 3521264583}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6826', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6827', 'nonce': 3521264583}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6830', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6831', 'nonce': 3521264583}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6828', 'nonce': 3521264583}, {'type': 'v1', 'addr': '172.21.15.183:6829', 'nonce': 3521264583}]}, 'public_addr': '172.21.15.183:6825/3521264583', 'cluster_addr': '172.21.15.183:6827/3521264583', 'heartbeat_back_addr': '172.21.15.183:6831/3521264583', 'heartbeat_front_addr': '172.21.15.183:6829/3521264583', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd3b1821c-1ed4-4286-b7de-030fda235fbf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6816', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6817', 'nonce': 2743896476}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6818', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6819', 'nonce': 2743896476}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6822', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6823', 'nonce': 2743896476}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6820', 'nonce': 2743896476}, {'type': 'v1', 'addr': '172.21.15.183:6821', 'nonce': 2743896476}]}, 'public_addr': '172.21.15.183:6817/2743896476', 'cluster_addr': '172.21.15.183:6819/2743896476', 'heartbeat_back_addr': '172.21.15.183:6823/2743896476', 'heartbeat_front_addr': '172.21.15.183:6821/2743896476', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.763716+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.782386+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.712986+0000', 'dead_epoch': 39}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.846199+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.735683+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.776861+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.767124+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-09T08:55:51.746641+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1d', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.24', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '3.39', 'mappings': [{'from': 7, 'to': 2}]}, {'pgid': '3.3c', 'mappings': [{'from': 5, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-09T08:57:43.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:57:43.392 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.392 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:43.393 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:43.389+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:43.488 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:43.489+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:43.529 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.529 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.552+0000 7f6d43e9e640 1 Processor -- start 2024-05-09T08:57:43.552 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.553+0000 7f6d43e9e640 1 -- start start 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d43e9e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d3c151e80 0x7f6d3c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d43e9e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6d3c071390 0x7f6d3c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d43e9e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3c15bfd0 con 0x7f6d3c073390 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d43e9e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c152260 con 0x7f6d3c151e80 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d43e9e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6d3c15b730 con 0x7f6d3c071390 2024-05-09T08:57:43.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d3c151e80 0x7f6d3c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6d3c151e80 0x7f6d3c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42000/0 (socket says 172.21.15.49:42000) 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 -- 172.21.15.49:0/2011585800 learned_addr learned my addr 172.21.15.49:0/2011585800 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d4243b640 1 --2- 172.21.15.49:0/2011585800 >> v2:172.21.15.183:3300/0 conn(0x7f6d3c071390 0x7f6d3c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 -- 172.21.15.49:0/2011585800 >> v2:172.21.15.183:3300/0 conn(0x7f6d3c071390 msgr2=0x7f6d3c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/2011585800 >> v2:172.21.15.183:3300/0 conn(0x7f6d3c071390 0x7f6d3c0cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 -- 172.21.15.49:0/2011585800 >> v1:172.21.15.183:6789/0 conn(0x7f6d3c073390 legacy=0x7f6d3c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.554+0000 7f6d41c3a640 1 -- 172.21.15.49:0/2011585800 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c15ba10 con 0x7f6d3c151e80 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.555+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/2011585800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c151e80 0x7f6d3c15b080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0c8380 tx=0x7f6d3802c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f4eab060e7e896 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.556+0000 7f6d40c38640 1 -- 172.21.15.49:0/2011585800 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38032780 con 0x7f6d3c151e80 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.556+0000 7f6d40c38640 1 -- 172.21.15.49:0/2011585800 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d38038070 con 0x7f6d3c151e80 2024-05-09T08:57:43.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.556+0000 7f6d40c38640 1 -- 172.21.15.49:0/2011585800 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38032b40 con 0x7f6d3c151e80 2024-05-09T08:57:43.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.556+0000 7f6d43e9e640 1 -- 172.21.15.49:0/2011585800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c151e80 msgr2=0x7f6d3c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.556+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/2011585800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c151e80 0x7f6d3c15b080 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0c8380 tx=0x7f6d3802c8c0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 -- 172.21.15.49:0/2011585800 shutdown_connections 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/2011585800 >> v2:172.21.15.183:3300/0 conn(0x7f6d3c071390 0x7f6d3c0cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/2011585800 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c151e80 0x7f6d3c15b080 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 -- 172.21.15.49:0/2011585800 >> 172.21.15.49:0/2011585800 conn(0x7f6d3c06eb60 msgr2=0x7f6d3c06ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 -- 172.21.15.49:0/2011585800 shutdown_connections 2024-05-09T08:57:43.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.557+0000 7f6d43e9e640 1 -- 172.21.15.49:0/2011585800 wait complete. 2024-05-09T08:57:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.559+0000 7f6d43e9e640 1 Processor -- start 2024-05-09T08:57:43.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.559+0000 7f6d43e9e640 1 -- start start 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.560+0000 7f6d43e9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 0x7f6d3c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.560+0000 7f6d43e9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d43e9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c151e80 0x7f6d3c1ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d43e9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c157f10 con 0x7f6d3c071390 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d41439640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d41439640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59678/0 (socket says 172.21.15.49:59678) 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d41439640 1 -- 172.21.15.49:0/4195064553 learned_addr learned my addr 172.21.15.49:0/4195064553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d4243b640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c151e80 0x7f6d3c1ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 0x7f6d3c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c157c30 con 0x7f6d3c151e80 2024-05-09T08:57:43.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.561+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c0718e0 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 -- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c151e80 msgr2=0x7f6d3c1ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c151e80 0x7f6d3c1ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 -- 172.21.15.49:0/4195064553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 msgr2=0x7f6d3c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 0x7f6d3c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c0c4270 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 0x7f6d3c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d41439640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c460 tx=0x7f6d2c00c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c016020 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c012870 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c0c4510 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.562+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1efa30 con 0x7f6d3c073390 2024-05-09T08:57:43.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.563+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c012c60 con 0x7f6d3c073390 2024-05-09T08:57:43.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.564+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c068c70 con 0x7f6d3c073390 2024-05-09T08:57:43.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.565+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d2c019020 con 0x7f6d3c073390 2024-05-09T08:57:43.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.565+0000 7f6d327fc640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 0x7f6d2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:43.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.565+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5857+0+0 (secure 0 0 0) 0x7f6d2c08ab10 con 0x7f6d3c073390 2024-05-09T08:57:43.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.565+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 0x7f6d2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:43.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.566+0000 7f6d41c3a640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 0x7f6d2006e320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d38037000 tx=0x7f6d3803e040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:43.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.569+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d2c020030 con 0x7f6d3c073390 2024-05-09T08:57:43.589 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:43.590+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:43.659 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.659 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:43.690 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:43.691+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:43.791 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:43.792+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:43.830 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.830 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.839 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:43.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.885+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d3c0c8f30 con 0x7f6d2006be60 2024-05-09T08:57:43.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.891+0000 7f6d327fc640 1 -- 172.21.15.49:0/4195064553 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252196 (secure 0 0 0) 0x7f6d3c0c8f30 con 0x7f6d2006be60 2024-05-09T08:57:43.894 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:57:43.895 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 msgr2=0x7f6d2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 0x7f6d2006e320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d38037000 tx=0x7f6d3803e040 comp rx=0 tx=0).stop 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 msgr2=0x7f6d3c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c460 tx=0x7f6d2c00c920 comp rx=0 tx=0).stop 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 shutdown_connections 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6d2006be60 0x7f6d2006e320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c151e80 0x7f6d3c1ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d3c073390 0x7f6d3c0ccac0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 --2- 172.21.15.49:0/4195064553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6d3c071390 0x7f6d3c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 >> 172.21.15.49:0/4195064553 conn(0x7f6d3c06eb60 msgr2=0x7f6d3c148230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 shutdown_connections 2024-05-09T08:57:43.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:43.896+0000 7f6d43e9e640 1 -- 172.21.15.49:0/4195064553 wait complete. 2024-05-09T08:57:43.898 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:43.893+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:43.912 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-05-09T08:57:42.686002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603536,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":381,"num_write_kb":2493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":178,"acting":178,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238440,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736280,"statfs":{"total":773094113280,"available":772849950720,"internally_reserved":0,"allocated":5763072,"data_stored":3090583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.023913"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588656+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:57:38.588656+0000","last_peered":"2024-05-09T08:57:38.588656+0000","last_clean":"2024-05-09T08:57:38.588656+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:57:38.588656+0000","last_undegraded":"2024-05-09T08:57:38.588656+0000","last_fullsized":"2024-05-09T08:57:38.588656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681299+0000","last_change":"2024-05-09T08:56:01.695567+0000","last_active":"2024-05-09T08:57:39.681299+0000","last_peered":"2024-05-09T08:57:39.681299+0000","last_clean":"2024-05-09T08:57:39.681299+0000","last_became_active":"2024-05-09T08:56:01.695246+0000","last_became_peered":"2024-05-09T08:56:01.695246+0000","last_unstale":"2024-05-09T08:57:39.681299+0000","last_undegraded":"2024-05-09T08:57:39.681299+0000","last_fullsized":"2024-05-09T08:57:39.681299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:18:51.651945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.681410+0000","last_change":"2024-05-09T08:57:16.404554+0000","last_active":"2024-05-09T08:57:39.681410+0000","last_peered":"2024-05-09T08:57:39.681410+0000","last_clean":"2024-05-09T08:57:12.974160+0000","last_became_active":"2024-05-09T08:57:16.404554+0000","last_became_peered":"2024-05-09T08:57:16.404554+0000","last_unstale":"2024-05-09T08:57:39.681410+0000","last_undegraded":"2024-05-09T08:57:39.681410+0000","last_fullsized":"2024-05-09T08:57:16.403888+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:10.186189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-05-09T08:57:37.668526+0000","last_change":"2024-05-09T08:57:16.402558+0000","last_active":"2024-05-09T08:57:37.668526+0000","last_peered":"2024-05-09T08:57:37.668526+0000","last_clean":"2024-05-09T08:56:00.667962+0000","last_became_active":"2024-05-09T08:57:16.402558+0000","last_became_peered":"2024-05-09T08:57:16.402558+0000","last_unstale":"2024-05-09T08:57:37.668526+0000","last_undegraded":"2024-05-09T08:57:37.668526+0000","last_fullsized":"2024-05-09T08:57:16.401425+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:36:05.502646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":68,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-05-09T08:57:38.588206+0000","last_change":"2024-05-09T08:57:16.407399+0000","last_active":"2024-05-09T08:57:38.588206+0000","last_peered":"2024-05-09T08:57:38.588206+0000","last_clean":"2024-05-09T08:57:12.383747+0000","last_became_active":"2024-05-09T08:57:16.407399+0000","last_became_peered":"2024-05-09T08:57:16.407399+0000","last_unstale":"2024-05-09T08:57:38.588206+0000","last_undegraded":"2024-05-09T08:57:38.588206+0000","last_fullsized":"2024-05-09T08:57:16.406708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:32:56.876243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688126+0000","last_change":"2024-05-09T08:56:01.690214+0000","last_active":"2024-05-09T08:57:39.688126+0000","last_peered":"2024-05-09T08:57:39.688126+0000","last_clean":"2024-05-09T08:57:39.688126+0000","last_became_active":"2024-05-09T08:56:01.689945+0000","last_became_peered":"2024-05-09T08:56:01.689945+0000","last_unstale":"2024-05-09T08:57:39.688126+0000","last_undegraded":"2024-05-09T08:57:39.688126+0000","last_fullsized":"2024-05-09T08:57:39.688126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:06:29.442415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":27,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.680007+0000","last_change":"2024-05-09T08:57:16.404800+0000","last_active":"2024-05-09T08:57:39.680007+0000","last_peered":"2024-05-09T08:57:39.680007+0000","last_clean":"2024-05-09T08:56:00.667962+0000","last_became_active":"2024-05-09T08:57:16.404800+0000","last_became_peered":"2024-05-09T08:57:16.404800+0000","last_unstale":"2024-05-09T08:57:39.680007+0000","last_undegraded":"2024-05-09T08:57:39.680007+0000","last_fullsized":"2024-05-09T08:57:16.403999+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:20.793829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667126+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:57:37.667126+0000","last_peered":"2024-05-09T08:57:37.667126+0000","last_clean":"2024-05-09T08:57:37.667126+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:57:37.667126+0000","last_undegraded":"2024-05-09T08:57:37.667126+0000","last_fullsized":"2024-05-09T08:57:37.667126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588698+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:57:38.588698+0000","last_peered":"2024-05-09T08:57:38.588698+0000","last_clean":"2024-05-09T08:57:38.588698+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:57:38.588698+0000","last_undegraded":"2024-05-09T08:57:38.588698+0000","last_fullsized":"2024-05-09T08:57:38.588698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122426+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:57:38.122426+0000","last_peered":"2024-05-09T08:57:38.122426+0000","last_clean":"2024-05-09T08:57:38.122426+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:57:38.122426+0000","last_undegraded":"2024-05-09T08:57:38.122426+0000","last_fullsized":"2024-05-09T08:57:38.122426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122158+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:57:38.122158+0000","last_peered":"2024-05-09T08:57:38.122158+0000","last_clean":"2024-05-09T08:57:38.122158+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:57:38.122158+0000","last_undegraded":"2024-05-09T08:57:38.122158+0000","last_fullsized":"2024-05-09T08:57:38.122158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.679007+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:57:39.679007+0000","last_peered":"2024-05-09T08:57:39.679007+0000","last_clean":"2024-05-09T08:57:39.679007+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:57:39.679007+0000","last_undegraded":"2024-05-09T08:57:39.679007+0000","last_fullsized":"2024-05-09T08:57:39.679007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.668194+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:57:37.668194+0000","last_peered":"2024-05-09T08:57:37.668194+0000","last_clean":"2024-05-09T08:57:37.668194+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:57:37.668194+0000","last_undegraded":"2024-05-09T08:57:37.668194+0000","last_fullsized":"2024-05-09T08:57:37.668194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680500+0000","last_change":"2024-05-09T08:56:01.691097+0000","last_active":"2024-05-09T08:57:39.680500+0000","last_peered":"2024-05-09T08:57:39.680500+0000","last_clean":"2024-05-09T08:57:39.680500+0000","last_became_active":"2024-05-09T08:56:01.690379+0000","last_became_peered":"2024-05-09T08:56:01.690379+0000","last_unstale":"2024-05-09T08:57:39.680500+0000","last_undegraded":"2024-05-09T08:57:39.680500+0000","last_fullsized":"2024-05-09T08:57:39.680500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:53:17.659170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680340+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:57:39.680340+0000","last_peered":"2024-05-09T08:57:39.680340+0000","last_clean":"2024-05-09T08:57:39.680340+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:57:39.680340+0000","last_undegraded":"2024-05-09T08:57:39.680340+0000","last_fullsized":"2024-05-09T08:57:39.680340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688074+0000","last_change":"2024-05-09T08:56:01.689525+0000","last_active":"2024-05-09T08:57:39.688074+0000","last_peered":"2024-05-09T08:57:39.688074+0000","last_clean":"2024-05-09T08:57:39.688074+0000","last_became_active":"2024-05-09T08:56:01.689221+0000","last_became_peered":"2024-05-09T08:56:01.689221+0000","last_unstale":"2024-05-09T08:57:39.688074+0000","last_undegraded":"2024-05-09T08:57:39.688074+0000","last_fullsized":"2024-05-09T08:57:39.688074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:30:29.907527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688560+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:57:39.688560+0000","last_peered":"2024-05-09T08:57:39.688560+0000","last_clean":"2024-05-09T08:57:39.688560+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:57:39.688560+0000","last_undegraded":"2024-05-09T08:57:39.688560+0000","last_fullsized":"2024-05-09T08:57:39.688560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-05-09T08:57:38.122384+0000","last_change":"2024-05-09T08:57:16.407306+0000","last_active":"2024-05-09T08:57:38.122384+0000","last_peered":"2024-05-09T08:57:38.122384+0000","last_clean":"2024-05-09T08:56:00.667962+0000","last_became_active":"2024-05-09T08:57:16.407306+0000","last_became_peered":"2024-05-09T08:57:16.407306+0000","last_unstale":"2024-05-09T08:57:38.122384+0000","last_undegraded":"2024-05-09T08:57:38.122384+0000","last_fullsized":"2024-05-09T08:57:16.406431+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:40:27.016568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":68,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-05-09T08:57:38.588671+0000","last_change":"2024-05-09T08:57:16.407731+0000","last_active":"2024-05-09T08:57:38.588671+0000","last_peered":"2024-05-09T08:57:38.588671+0000","last_clean":"2024-05-09T08:57:12.384298+0000","last_became_active":"2024-05-09T08:57:16.407731+0000","last_became_peered":"2024-05-09T08:57:16.407731+0000","last_unstale":"2024-05-09T08:57:38.588671+0000","last_undegraded":"2024-05-09T08:57:38.588671+0000","last_fullsized":"2024-05-09T08:57:16.407147+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:48:19.885549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.121907+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:57:38.121907+0000","last_peered":"2024-05-09T08:57:38.121907+0000","last_clean":"2024-05-09T08:57:38.121907+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:57:38.121907+0000","last_undegraded":"2024-05-09T08:57:38.121907+0000","last_fullsized":"2024-05-09T08:57:38.121907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680782+0000","last_change":"2024-05-09T08:56:01.699930+0000","last_active":"2024-05-09T08:57:39.680782+0000","last_peered":"2024-05-09T08:57:39.680782+0000","last_clean":"2024-05-09T08:57:39.680782+0000","last_became_active":"2024-05-09T08:56:01.699592+0000","last_became_peered":"2024-05-09T08:56:01.699592+0000","last_unstale":"2024-05-09T08:57:39.680782+0000","last_undegraded":"2024-05-09T08:57:39.680782+0000","last_fullsized":"2024-05-09T08:57:39.680782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:50.714635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688175+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:57:39.688175+0000","last_peered":"2024-05-09T08:57:39.688175+0000","last_clean":"2024-05-09T08:57:39.688175+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:57:39.688175+0000","last_undegraded":"2024-05-09T08:57:39.688175+0000","last_fullsized":"2024-05-09T08:57:39.688175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667456+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:57:37.667456+0000","last_peered":"2024-05-09T08:57:37.667456+0000","last_clean":"2024-05-09T08:57:37.667456+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:57:37.667456+0000","last_undegraded":"2024-05-09T08:57:37.667456+0000","last_fullsized":"2024-05-09T08:57:37.667456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688123+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:57:39.688123+0000","last_peered":"2024-05-09T08:57:39.688123+0000","last_clean":"2024-05-09T08:57:39.688123+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:57:39.688123+0000","last_undegraded":"2024-05-09T08:57:39.688123+0000","last_fullsized":"2024-05-09T08:57:39.688123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.668918+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:57:37.668918+0000","last_peered":"2024-05-09T08:57:37.668918+0000","last_clean":"2024-05-09T08:57:37.668918+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:57:37.668918+0000","last_undegraded":"2024-05-09T08:57:37.668918+0000","last_fullsized":"2024-05-09T08:57:37.668918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122485+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:57:38.122485+0000","last_peered":"2024-05-09T08:57:38.122485+0000","last_clean":"2024-05-09T08:57:38.122485+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:57:38.122485+0000","last_undegraded":"2024-05-09T08:57:38.122485+0000","last_fullsized":"2024-05-09T08:57:38.122485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680937+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:57:39.680937+0000","last_peered":"2024-05-09T08:57:39.680937+0000","last_clean":"2024-05-09T08:57:39.680937+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:57:39.680937+0000","last_undegraded":"2024-05-09T08:57:39.680937+0000","last_fullsized":"2024-05-09T08:57:39.680937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":56,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680796+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:57:39.680796+0000","last_peered":"2024-05-09T08:57:39.680796+0000","last_clean":"2024-05-09T08:57:39.680796+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:57:39.680796+0000","last_undegraded":"2024-05-09T08:57:39.680796+0000","last_fullsized":"2024-05-09T08:57:39.680796+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680865+0000","last_change":"2024-05-09T08:56:01.701050+0000","last_active":"2024-05-09T08:57:39.680865+0000","last_peered":"2024-05-09T08:57:39.680865+0000","last_clean":"2024-05-09T08:57:39.680865+0000","last_became_active":"2024-05-09T08:56:01.700717+0000","last_became_peered":"2024-05-09T08:56:01.700717+0000","last_unstale":"2024-05-09T08:57:39.680865+0000","last_undegraded":"2024-05-09T08:57:39.680865+0000","last_fullsized":"2024-05-09T08:57:39.680865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:43.302092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688225+0000","last_change":"2024-05-09T08:56:01.695168+0000","last_active":"2024-05-09T08:57:39.688225+0000","last_peered":"2024-05-09T08:57:39.688225+0000","last_clean":"2024-05-09T08:57:39.688225+0000","last_became_active":"2024-05-09T08:56:01.694754+0000","last_became_peered":"2024-05-09T08:56:01.694754+0000","last_unstale":"2024-05-09T08:57:39.688225+0000","last_undegraded":"2024-05-09T08:57:39.688225+0000","last_fullsized":"2024-05-09T08:57:39.688225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:53:59.108726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.688109+0000","last_change":"2024-05-09T08:57:16.405128+0000","last_active":"2024-05-09T08:57:39.688109+0000","last_peered":"2024-05-09T08:57:39.688109+0000","last_clean":"2024-05-09T08:56:00.667962+0000","last_became_active":"2024-05-09T08:57:16.405128+0000","last_became_peered":"2024-05-09T08:57:16.405128+0000","last_unstale":"2024-05-09T08:57:39.688109+0000","last_undegraded":"2024-05-09T08:57:39.688109+0000","last_fullsized":"2024-05-09T08:57:16.403633+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:17:16.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.121990+0000","last_change":"2024-05-09T08:56:01.696670+0000","last_active":"2024-05-09T08:57:38.121990+0000","last_peered":"2024-05-09T08:57:38.121990+0000","last_clean":"2024-05-09T08:57:38.121990+0000","last_became_active":"2024-05-09T08:56:01.696057+0000","last_became_peered":"2024-05-09T08:56:01.696057+0000","last_unstale":"2024-05-09T08:57:38.121990+0000","last_undegraded":"2024-05-09T08:57:38.121990+0000","last_fullsized":"2024-05-09T08:57:38.121990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:52:21.328538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.678703+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:57:39.678703+0000","last_peered":"2024-05-09T08:57:39.678703+0000","last_clean":"2024-05-09T08:57:39.678703+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:57:39.678703+0000","last_undegraded":"2024-05-09T08:57:39.678703+0000","last_fullsized":"2024-05-09T08:57:39.678703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.668360+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:57:37.668360+0000","last_peered":"2024-05-09T08:57:37.668360+0000","last_clean":"2024-05-09T08:57:37.668360+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:57:37.668360+0000","last_undegraded":"2024-05-09T08:57:37.668360+0000","last_fullsized":"2024-05-09T08:57:37.668360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680950+0000","last_change":"2024-05-09T08:56:01.689664+0000","last_active":"2024-05-09T08:57:39.680950+0000","last_peered":"2024-05-09T08:57:39.680950+0000","last_clean":"2024-05-09T08:57:39.680950+0000","last_became_active":"2024-05-09T08:56:01.689376+0000","last_became_peered":"2024-05-09T08:56:01.689376+0000","last_unstale":"2024-05-09T08:57:39.680950+0000","last_undegraded":"2024-05-09T08:57:39.680950+0000","last_fullsized":"2024-05-09T08:57:39.680950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:28.923033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122037+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:57:38.122037+0000","last_peered":"2024-05-09T08:57:38.122037+0000","last_clean":"2024-05-09T08:57:38.122037+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:57:38.122037+0000","last_undegraded":"2024-05-09T08:57:38.122037+0000","last_fullsized":"2024-05-09T08:57:38.122037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":71,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667730+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:57:37.667730+0000","last_peered":"2024-05-09T08:57:37.667730+0000","last_clean":"2024-05-09T08:57:37.667730+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:57:37.667730+0000","last_undegraded":"2024-05-09T08:57:37.667730+0000","last_fullsized":"2024-05-09T08:57:37.667730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667231+0000","last_change":"2024-05-09T08:56:01.696195+0000","last_active":"2024-05-09T08:57:37.667231+0000","last_peered":"2024-05-09T08:57:37.667231+0000","last_clean":"2024-05-09T08:57:37.667231+0000","last_became_active":"2024-05-09T08:56:01.695892+0000","last_became_peered":"2024-05-09T08:56:01.695892+0000","last_unstale":"2024-05-09T08:57:37.667231+0000","last_undegraded":"2024-05-09T08:57:37.667231+0000","last_fullsized":"2024-05-09T08:57:37.667231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:45:24.382635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688229+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:57:39.688229+0000","last_peered":"2024-05-09T08:57:39.688229+0000","last_clean":"2024-05-09T08:57:39.688229+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:57:39.688229+0000","last_undegraded":"2024-05-09T08:57:39.688229+0000","last_fullsized":"2024-05-09T08:57:39.688229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680252+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:57:39.680252+0000","last_peered":"2024-05-09T08:57:39.680252+0000","last_clean":"2024-05-09T08:57:39.680252+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:57:39.680252+0000","last_undegraded":"2024-05-09T08:57:39.680252+0000","last_fullsized":"2024-05-09T08:57:39.680252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"20'5","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122307+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:57:38.122307+0000","last_peered":"2024-05-09T08:57:38.122307+0000","last_clean":"2024-05-09T08:57:38.122307+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:57:38.122307+0000","last_undegraded":"2024-05-09T08:57:38.122307+0000","last_fullsized":"2024-05-09T08:57:38.122307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688470+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:57:39.688470+0000","last_peered":"2024-05-09T08:57:39.688470+0000","last_clean":"2024-05-09T08:57:39.688470+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:57:39.688470+0000","last_undegraded":"2024-05-09T08:57:39.688470+0000","last_fullsized":"2024-05-09T08:57:39.688470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688479+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:57:39.688479+0000","last_peered":"2024-05-09T08:57:39.688479+0000","last_clean":"2024-05-09T08:57:39.688479+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:57:39.688479+0000","last_undegraded":"2024-05-09T08:57:39.688479+0000","last_fullsized":"2024-05-09T08:57:39.688479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681620+0000","last_change":"2024-05-09T08:56:01.690992+0000","last_active":"2024-05-09T08:57:39.681620+0000","last_peered":"2024-05-09T08:57:39.681620+0000","last_clean":"2024-05-09T08:57:39.681620+0000","last_became_active":"2024-05-09T08:56:01.690663+0000","last_became_peered":"2024-05-09T08:56:01.690663+0000","last_unstale":"2024-05-09T08:57:39.681620+0000","last_undegraded":"2024-05-09T08:57:39.681620+0000","last_fullsized":"2024-05-09T08:57:39.681620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:25:11.760446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'5","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681055+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:57:39.681055+0000","last_peered":"2024-05-09T08:57:39.681055+0000","last_clean":"2024-05-09T08:57:39.681055+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:57:39.681055+0000","last_undegraded":"2024-05-09T08:57:39.681055+0000","last_fullsized":"2024-05-09T08:57:39.681055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.679035+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:57:39.679035+0000","last_peered":"2024-05-09T08:57:39.679035+0000","last_clean":"2024-05-09T08:57:39.679035+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:57:39.679035+0000","last_undegraded":"2024-05-09T08:57:39.679035+0000","last_fullsized":"2024-05-09T08:57:39.679035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"40'16","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122125+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:57:38.122125+0000","last_peered":"2024-05-09T08:57:38.122125+0000","last_clean":"2024-05-09T08:57:38.122125+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:57:38.122125+0000","last_undegraded":"2024-05-09T08:57:38.122125+0000","last_fullsized":"2024-05-09T08:57:38.122125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6490,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588018+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:57:38.588018+0000","last_peered":"2024-05-09T08:57:38.588018+0000","last_clean":"2024-05-09T08:57:38.588018+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:57:38.588018+0000","last_undegraded":"2024-05-09T08:57:38.588018+0000","last_fullsized":"2024-05-09T08:57:38.588018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681038+0000","last_change":"2024-05-09T08:56:00.685489+0000","last_active":"2024-05-09T08:57:39.681038+0000","last_peered":"2024-05-09T08:57:39.681038+0000","last_clean":"2024-05-09T08:57:39.681038+0000","last_became_active":"2024-05-09T08:56:00.685139+0000","last_became_peered":"2024-05-09T08:56:00.685139+0000","last_unstale":"2024-05-09T08:57:39.681038+0000","last_undegraded":"2024-05-09T08:57:39.681038+0000","last_fullsized":"2024-05-09T08:57:39.681038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:29.207862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680590+0000","last_change":"2024-05-09T08:56:01.699567+0000","last_active":"2024-05-09T08:57:39.680590+0000","last_peered":"2024-05-09T08:57:39.680590+0000","last_clean":"2024-05-09T08:57:39.680590+0000","last_became_active":"2024-05-09T08:56:01.699225+0000","last_became_peered":"2024-05-09T08:56:01.699225+0000","last_unstale":"2024-05-09T08:57:39.680590+0000","last_undegraded":"2024-05-09T08:57:39.680590+0000","last_fullsized":"2024-05-09T08:57:39.680590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:51:44.694925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"20'5","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688283+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:57:39.688283+0000","last_peered":"2024-05-09T08:57:39.688283+0000","last_clean":"2024-05-09T08:57:39.688283+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:57:39.688283+0000","last_undegraded":"2024-05-09T08:57:39.688283+0000","last_fullsized":"2024-05-09T08:57:39.688283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588014+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:57:38.588014+0000","last_peered":"2024-05-09T08:57:38.588014+0000","last_clean":"2024-05-09T08:57:38.588014+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:57:38.588014+0000","last_undegraded":"2024-05-09T08:57:38.588014+0000","last_fullsized":"2024-05-09T08:57:38.588014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122164+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:57:38.122164+0000","last_peered":"2024-05-09T08:57:38.122164+0000","last_clean":"2024-05-09T08:57:38.122164+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:57:38.122164+0000","last_undegraded":"2024-05-09T08:57:38.122164+0000","last_fullsized":"2024-05-09T08:57:38.122164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122043+0000","last_change":"2024-05-09T08:56:01.696392+0000","last_active":"2024-05-09T08:57:38.122043+0000","last_peered":"2024-05-09T08:57:38.122043+0000","last_clean":"2024-05-09T08:57:38.122043+0000","last_became_active":"2024-05-09T08:56:01.694664+0000","last_became_peered":"2024-05-09T08:56:01.694664+0000","last_unstale":"2024-05-09T08:57:38.122043+0000","last_undegraded":"2024-05-09T08:57:38.122043+0000","last_fullsized":"2024-05-09T08:57:38.122043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:08:38.219360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":49,"reported_epoch":49,"state":"active+undersized+degraded","last_fresh":"2024-05-09T08:57:39.680007+0000","last_change":"2024-05-09T08:57:16.406945+0000","last_active":"2024-05-09T08:57:39.680007+0000","last_peered":"2024-05-09T08:57:39.680007+0000","last_clean":"2024-05-09T08:56:11.729134+0000","last_became_active":"2024-05-09T08:57:16.406945+0000","last_became_peered":"2024-05-09T08:57:16.406945+0000","last_unstale":"2024-05-09T08:57:39.680007+0000","last_undegraded":"2024-05-09T08:57:16.406009+0000","last_fullsized":"2024-05-09T08:57:16.405891+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:37:25.958129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":8}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588011+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:57:38.588011+0000","last_peered":"2024-05-09T08:57:38.588011+0000","last_clean":"2024-05-09T08:57:38.588011+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:57:38.588011+0000","last_undegraded":"2024-05-09T08:57:38.588011+0000","last_fullsized":"2024-05-09T08:57:38.588011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.679150+0000","last_change":"2024-05-09T08:56:00.688084+0000","last_active":"2024-05-09T08:57:39.679150+0000","last_peered":"2024-05-09T08:57:39.679150+0000","last_clean":"2024-05-09T08:57:39.679150+0000","last_became_active":"2024-05-09T08:56:00.687710+0000","last_became_peered":"2024-05-09T08:56:00.687710+0000","last_unstale":"2024-05-09T08:57:39.679150+0000","last_undegraded":"2024-05-09T08:57:39.679150+0000","last_fullsized":"2024-05-09T08:57:39.679150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:02:10.653629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588038+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:57:38.588038+0000","last_peered":"2024-05-09T08:57:38.588038+0000","last_clean":"2024-05-09T08:57:38.588038+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:57:38.588038+0000","last_undegraded":"2024-05-09T08:57:38.588038+0000","last_fullsized":"2024-05-09T08:57:38.588038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":75,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-05-09T08:57:37.669186+0000","last_change":"2024-05-09T08:57:16.403020+0000","last_active":"2024-05-09T08:57:37.669186+0000","last_peered":"2024-05-09T08:57:37.669186+0000","last_clean":"2024-05-09T08:57:12.973295+0000","last_became_active":"2024-05-09T08:57:16.403020+0000","last_became_peered":"2024-05-09T08:57:16.403020+0000","last_unstale":"2024-05-09T08:57:37.669186+0000","last_undegraded":"2024-05-09T08:57:16.401838+0000","last_fullsized":"2024-05-09T08:57:16.401758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:19:55.951459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588074+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:57:38.588074+0000","last_peered":"2024-05-09T08:57:38.588074+0000","last_clean":"2024-05-09T08:57:38.588074+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:57:38.588074+0000","last_undegraded":"2024-05-09T08:57:38.588074+0000","last_fullsized":"2024-05-09T08:57:38.588074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":73,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681192+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:57:39.681192+0000","last_peered":"2024-05-09T08:57:39.681192+0000","last_clean":"2024-05-09T08:57:39.681192+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:57:39.681192+0000","last_undegraded":"2024-05-09T08:57:39.681192+0000","last_fullsized":"2024-05-09T08:57:39.681192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688301+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:57:39.688301+0000","last_peered":"2024-05-09T08:57:39.688301+0000","last_clean":"2024-05-09T08:57:39.688301+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:57:39.688301+0000","last_undegraded":"2024-05-09T08:57:39.688301+0000","last_fullsized":"2024-05-09T08:57:39.688301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":43,"reported_epoch":49,"state":"active+undersized+degraded","last_fresh":"2024-05-09T08:57:39.688361+0000","last_change":"2024-05-09T08:57:16.405444+0000","last_active":"2024-05-09T08:57:39.688361+0000","last_peered":"2024-05-09T08:57:39.688361+0000","last_clean":"2024-05-09T08:56:23.750621+0000","last_became_active":"2024-05-09T08:57:16.405444+0000","last_became_peered":"2024-05-09T08:57:16.405444+0000","last_unstale":"2024-05-09T08:57:39.688361+0000","last_undegraded":"2024-05-09T08:57:16.403847+0000","last_fullsized":"2024-05-09T08:57:16.403770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:20.941831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":3}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588557+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:57:38.588557+0000","last_peered":"2024-05-09T08:57:38.588557+0000","last_clean":"2024-05-09T08:57:38.588557+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:57:38.588557+0000","last_undegraded":"2024-05-09T08:57:38.588557+0000","last_fullsized":"2024-05-09T08:57:38.588557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588370+0000","last_change":"2024-05-09T08:55:57.824074+0000","last_active":"2024-05-09T08:57:38.588370+0000","last_peered":"2024-05-09T08:57:38.588370+0000","last_clean":"2024-05-09T08:57:38.588370+0000","last_became_active":"2024-05-09T08:55:57.823069+0000","last_became_peered":"2024-05-09T08:55:57.823069+0000","last_unstale":"2024-05-09T08:57:38.588370+0000","last_undegraded":"2024-05-09T08:57:38.588370+0000","last_fullsized":"2024-05-09T08:57:38.588370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:52:06.700739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":79,"reported_epoch":49,"state":"active+undersized+degraded","last_fresh":"2024-05-09T08:57:39.688171+0000","last_change":"2024-05-09T08:57:16.405166+0000","last_active":"2024-05-09T08:57:39.688171+0000","last_peered":"2024-05-09T08:57:39.688171+0000","last_clean":"2024-05-09T08:57:12.782993+0000","last_became_active":"2024-05-09T08:57:16.405166+0000","last_became_peered":"2024-05-09T08:57:16.405166+0000","last_unstale":"2024-05-09T08:57:39.688171+0000","last_undegraded":"2024-05-09T08:57:16.404038+0000","last_fullsized":"2024-05-09T08:57:16.403961+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:03:27.737989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":5}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667965+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:57:37.667965+0000","last_peered":"2024-05-09T08:57:37.667965+0000","last_clean":"2024-05-09T08:57:37.667965+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:57:37.667965+0000","last_undegraded":"2024-05-09T08:57:37.667965+0000","last_fullsized":"2024-05-09T08:57:37.667965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588371+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:57:38.588371+0000","last_peered":"2024-05-09T08:57:38.588371+0000","last_clean":"2024-05-09T08:57:38.588371+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:57:38.588371+0000","last_undegraded":"2024-05-09T08:57:38.588371+0000","last_fullsized":"2024-05-09T08:57:38.588371+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680179+0000","last_change":"2024-05-09T08:56:01.690061+0000","last_active":"2024-05-09T08:57:39.680179+0000","last_peered":"2024-05-09T08:57:39.680179+0000","last_clean":"2024-05-09T08:57:39.680179+0000","last_became_active":"2024-05-09T08:56:01.689478+0000","last_became_peered":"2024-05-09T08:56:01.689478+0000","last_unstale":"2024-05-09T08:57:39.680179+0000","last_undegraded":"2024-05-09T08:57:39.680179+0000","last_fullsized":"2024-05-09T08:57:39.680179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:31:35.761126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'5","reported_seq":41,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-05-09T08:57:37.667545+0000","last_change":"2024-05-09T08:57:16.402557+0000","last_active":"2024-05-09T08:57:37.667545+0000","last_peered":"2024-05-09T08:57:37.667545+0000","last_clean":"2024-05-09T08:56:17.735438+0000","last_became_active":"2024-05-09T08:57:16.402557+0000","last_became_peered":"2024-05-09T08:57:16.402557+0000","last_unstale":"2024-05-09T08:57:37.667545+0000","last_undegraded":"2024-05-09T08:57:16.401379+0000","last_fullsized":"2024-05-09T08:57:16.401285+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:13.270642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":3}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.679825+0000","last_change":"2024-05-09T08:57:16.406642+0000","last_active":"2024-05-09T08:57:39.679825+0000","last_peered":"2024-05-09T08:57:39.679825+0000","last_clean":"2024-05-09T08:57:12.489600+0000","last_became_active":"2024-05-09T08:57:16.406642+0000","last_became_peered":"2024-05-09T08:57:16.406642+0000","last_unstale":"2024-05-09T08:57:39.679825+0000","last_undegraded":"2024-05-09T08:57:39.679825+0000","last_fullsized":"2024-05-09T08:57:16.405840+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:50:27.982658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688321+0000","last_change":"2024-05-09T08:56:00.690895+0000","last_active":"2024-05-09T08:57:39.688321+0000","last_peered":"2024-05-09T08:57:39.688321+0000","last_clean":"2024-05-09T08:57:39.688321+0000","last_became_active":"2024-05-09T08:56:00.690569+0000","last_became_peered":"2024-05-09T08:56:00.690569+0000","last_unstale":"2024-05-09T08:57:39.688321+0000","last_undegraded":"2024-05-09T08:57:39.688321+0000","last_fullsized":"2024-05-09T08:57:39.688321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:08.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.678952+0000","last_change":"2024-05-09T08:56:01.689758+0000","last_active":"2024-05-09T08:57:39.678952+0000","last_peered":"2024-05-09T08:57:39.678952+0000","last_clean":"2024-05-09T08:57:39.678952+0000","last_became_active":"2024-05-09T08:56:01.689427+0000","last_became_peered":"2024-05-09T08:56:01.689427+0000","last_unstale":"2024-05-09T08:57:39.678952+0000","last_undegraded":"2024-05-09T08:57:39.678952+0000","last_fullsized":"2024-05-09T08:57:39.678952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:26:09.963589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681054+0000","last_change":"2024-05-09T08:56:00.690459+0000","last_active":"2024-05-09T08:57:39.681054+0000","last_peered":"2024-05-09T08:57:39.681054+0000","last_clean":"2024-05-09T08:57:39.681054+0000","last_became_active":"2024-05-09T08:56:00.690131+0000","last_became_peered":"2024-05-09T08:56:00.690131+0000","last_unstale":"2024-05-09T08:57:39.681054+0000","last_undegraded":"2024-05-09T08:57:39.681054+0000","last_fullsized":"2024-05-09T08:57:39.681054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:15.670384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688442+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:57:39.688442+0000","last_peered":"2024-05-09T08:57:39.688442+0000","last_clean":"2024-05-09T08:57:39.688442+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:57:39.688442+0000","last_undegraded":"2024-05-09T08:57:39.688442+0000","last_fullsized":"2024-05-09T08:57:39.688442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588679+0000","last_change":"2024-05-09T08:56:00.696357+0000","last_active":"2024-05-09T08:57:38.588679+0000","last_peered":"2024-05-09T08:57:38.588679+0000","last_clean":"2024-05-09T08:57:38.588679+0000","last_became_active":"2024-05-09T08:56:00.696016+0000","last_became_peered":"2024-05-09T08:56:00.696016+0000","last_unstale":"2024-05-09T08:57:38.588679+0000","last_undegraded":"2024-05-09T08:57:38.588679+0000","last_fullsized":"2024-05-09T08:57:38.588679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:49:10.219794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":70,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.680882+0000","last_change":"2024-05-09T08:57:16.407001+0000","last_active":"2024-05-09T08:57:39.680882+0000","last_peered":"2024-05-09T08:57:39.680882+0000","last_clean":"2024-05-09T08:57:12.489768+0000","last_became_active":"2024-05-09T08:57:16.407001+0000","last_became_peered":"2024-05-09T08:57:16.407001+0000","last_unstale":"2024-05-09T08:57:39.680882+0000","last_undegraded":"2024-05-09T08:57:39.680882+0000","last_fullsized":"2024-05-09T08:57:16.406230+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:48.700312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"42'10","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681398+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:57:39.681398+0000","last_peered":"2024-05-09T08:57:39.681398+0000","last_clean":"2024-05-09T08:57:39.681398+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:57:39.681398+0000","last_undegraded":"2024-05-09T08:57:39.681398+0000","last_fullsized":"2024-05-09T08:57:39.681398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.669010+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:57:37.669010+0000","last_peered":"2024-05-09T08:57:37.669010+0000","last_clean":"2024-05-09T08:57:37.669010+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:57:37.669010+0000","last_undegraded":"2024-05-09T08:57:37.669010+0000","last_fullsized":"2024-05-09T08:57:37.669010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'7","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.122356+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:57:38.122356+0000","last_peered":"2024-05-09T08:57:38.122356+0000","last_clean":"2024-05-09T08:57:38.122356+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:57:38.122356+0000","last_undegraded":"2024-05-09T08:57:38.122356+0000","last_fullsized":"2024-05-09T08:57:38.122356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681243+0000","last_change":"2024-05-09T08:56:01.687683+0000","last_active":"2024-05-09T08:57:39.681243+0000","last_peered":"2024-05-09T08:57:39.681243+0000","last_clean":"2024-05-09T08:57:39.681243+0000","last_became_active":"2024-05-09T08:56:01.687392+0000","last_became_peered":"2024-05-09T08:56:01.687392+0000","last_unstale":"2024-05-09T08:57:39.681243+0000","last_undegraded":"2024-05-09T08:57:39.681243+0000","last_fullsized":"2024-05-09T08:57:39.681243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:46:14.659211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588269+0000","last_change":"2024-05-09T08:56:00.695126+0000","last_active":"2024-05-09T08:57:38.588269+0000","last_peered":"2024-05-09T08:57:38.588269+0000","last_clean":"2024-05-09T08:57:38.588269+0000","last_became_active":"2024-05-09T08:56:00.694933+0000","last_became_peered":"2024-05-09T08:56:00.694933+0000","last_unstale":"2024-05-09T08:57:38.588269+0000","last_undegraded":"2024-05-09T08:57:38.588269+0000","last_fullsized":"2024-05-09T08:57:38.588269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:04:56.312898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":72,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.678749+0000","last_change":"2024-05-09T08:57:16.404807+0000","last_active":"2024-05-09T08:57:39.678749+0000","last_peered":"2024-05-09T08:57:39.678749+0000","last_clean":"2024-05-09T08:57:12.974542+0000","last_became_active":"2024-05-09T08:57:16.404807+0000","last_became_peered":"2024-05-09T08:57:16.404807+0000","last_unstale":"2024-05-09T08:57:39.678749+0000","last_undegraded":"2024-05-09T08:57:39.678749+0000","last_fullsized":"2024-05-09T08:57:16.403871+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:16:37.086409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":77,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680424+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:57:39.680424+0000","last_peered":"2024-05-09T08:57:39.680424+0000","last_clean":"2024-05-09T08:57:39.680424+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:57:39.680424+0000","last_undegraded":"2024-05-09T08:57:39.680424+0000","last_fullsized":"2024-05-09T08:57:39.680424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.667313+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:57:37.667313+0000","last_peered":"2024-05-09T08:57:37.667313+0000","last_clean":"2024-05-09T08:57:37.667313+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:57:37.667313+0000","last_undegraded":"2024-05-09T08:57:37.667313+0000","last_fullsized":"2024-05-09T08:57:37.667313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":74,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.680685+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:57:39.680685+0000","last_peered":"2024-05-09T08:57:39.680685+0000","last_clean":"2024-05-09T08:57:39.680685+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:57:39.680685+0000","last_undegraded":"2024-05-09T08:57:39.680685+0000","last_fullsized":"2024-05-09T08:57:39.680685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.668276+0000","last_change":"2024-05-09T08:56:01.695224+0000","last_active":"2024-05-09T08:57:37.668276+0000","last_peered":"2024-05-09T08:57:37.668276+0000","last_clean":"2024-05-09T08:57:37.668276+0000","last_became_active":"2024-05-09T08:56:01.694840+0000","last_became_peered":"2024-05-09T08:56:01.694840+0000","last_unstale":"2024-05-09T08:57:37.668276+0000","last_undegraded":"2024-05-09T08:57:37.668276+0000","last_fullsized":"2024-05-09T08:57:37.668276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:21:23.245591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'4","reported_seq":72,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588400+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:57:38.588400+0000","last_peered":"2024-05-09T08:57:38.588400+0000","last_clean":"2024-05-09T08:57:38.588400+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:57:38.588400+0000","last_undegraded":"2024-05-09T08:57:38.588400+0000","last_fullsized":"2024-05-09T08:57:38.588400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.121899+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:57:38.121899+0000","last_peered":"2024-05-09T08:57:38.121899+0000","last_clean":"2024-05-09T08:57:38.121899+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:57:38.121899+0000","last_undegraded":"2024-05-09T08:57:38.121899+0000","last_fullsized":"2024-05-09T08:57:38.121899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":27,"reported_epoch":49,"state":"active+undersized","last_fresh":"2024-05-09T08:57:39.678892+0000","last_change":"2024-05-09T08:57:16.405230+0000","last_active":"2024-05-09T08:57:39.678892+0000","last_peered":"2024-05-09T08:57:39.678892+0000","last_clean":"2024-05-09T08:56:00.667962+0000","last_became_active":"2024-05-09T08:57:16.405230+0000","last_became_peered":"2024-05-09T08:57:16.405230+0000","last_unstale":"2024-05-09T08:57:39.678892+0000","last_undegraded":"2024-05-09T08:57:39.678892+0000","last_fullsized":"2024-05-09T08:57:16.404542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:31.143369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588423+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:57:38.588423+0000","last_peered":"2024-05-09T08:57:38.588423+0000","last_clean":"2024-05-09T08:57:38.588423+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:57:38.588423+0000","last_undegraded":"2024-05-09T08:57:38.588423+0000","last_fullsized":"2024-05-09T08:57:38.588423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.688208+0000","last_change":"2024-05-09T08:56:01.687651+0000","last_active":"2024-05-09T08:57:39.688208+0000","last_peered":"2024-05-09T08:57:39.688208+0000","last_clean":"2024-05-09T08:57:39.688208+0000","last_became_active":"2024-05-09T08:56:01.687370+0000","last_became_peered":"2024-05-09T08:56:01.687370+0000","last_unstale":"2024-05-09T08:57:39.688208+0000","last_undegraded":"2024-05-09T08:57:39.688208+0000","last_fullsized":"2024-05-09T08:57:39.688208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:22:10.429469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'12","reported_seq":80,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:38.588563+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:57:38.588563+0000","last_peered":"2024-05-09T08:57:38.588563+0000","last_clean":"2024-05-09T08:57:38.588563+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:57:38.588563+0000","last_undegraded":"2024-05-09T08:57:38.588563+0000","last_fullsized":"2024-05-09T08:57:38.588563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681619+0000","last_change":"2024-05-09T08:56:01.690943+0000","last_active":"2024-05-09T08:57:39.681619+0000","last_peered":"2024-05-09T08:57:39.681619+0000","last_clean":"2024-05-09T08:57:39.681619+0000","last_became_active":"2024-05-09T08:56:01.690450+0000","last_became_peered":"2024-05-09T08:56:01.690450+0000","last_unstale":"2024-05-09T08:57:39.681619+0000","last_undegraded":"2024-05-09T08:57:39.681619+0000","last_fullsized":"2024-05-09T08:57:39.681619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:22:11.436232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":70,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.669366+0000","last_change":"2024-05-09T08:56:00.696264+0000","last_active":"2024-05-09T08:57:37.669366+0000","last_peered":"2024-05-09T08:57:37.669366+0000","last_clean":"2024-05-09T08:57:37.669366+0000","last_became_active":"2024-05-09T08:56:00.695917+0000","last_became_peered":"2024-05-09T08:56:00.695917+0000","last_unstale":"2024-05-09T08:57:37.669366+0000","last_undegraded":"2024-05-09T08:57:37.669366+0000","last_fullsized":"2024-05-09T08:57:37.669366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-09T08:57:37.669100+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:57:37.669100+0000","last_peered":"2024-05-09T08:57:37.669100+0000","last_clean":"2024-05-09T08:57:37.669100+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:57:37.669100+0000","last_undegraded":"2024-05-09T08:57:37.669100+0000","last_fullsized":"2024-05-09T08:57:37.669100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":83,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-05-09T08:57:39.681748+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:57:39.681748+0000","last_peered":"2024-05-09T08:57:39.681748+0000","last_clean":"2024-05-09T08:57:39.681748+0000","last_became_active":"2024-05-09T08:56:00.693647+0000","last_became_peered":"2024-05-09T08:56:00.693647+0000","last_unstale":"2024-05-09T08:57:39.681748+0000","last_undegraded":"2024-05-09T08:57:39.681748+0000","last_fullsized":"2024-05-09T08:57:39.681748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:15.879047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":117,"acting":117,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":13168,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":26336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":191,"ondisk_log_size":191,"up":59,"acting":59,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":833520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":248076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":610304,"data_stored":250785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":248183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":585728,"data_stored":248366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":446464,"data_stored":175173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31756,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340084,"statfs":{"total":96636764160,"available":96604246016,"internally_reserved":0,"allocated":589824,"data_stored":248385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1183744,"data_stored":838095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":7570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":7280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:57:43.917 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [5, 6, 1, 7] 2024-05-09T08:57:43.917 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '5', '6', '1', '7'] 2024-05-09T08:57:43.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 5 6 1 7 2024-05-09T08:57:43.953 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:43.953 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:43.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:44.004 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:44.005+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.094+0000 7f6b19f61640 1 Processor -- start 2024-05-09T08:57:44.094 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.094+0000 7f6b19f61640 1 -- start start 2024-05-09T08:57:44.094 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:44.095+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:44.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b141592c0 0x7f6b1406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 0x7f6b14074840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b14074d80 0x7f6b14162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1406b740 con 0x7f6b140c8520 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6b140ca3e0 con 0x7f6b14074d80 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b19f61640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6b140ca680 con 0x7f6b141592c0 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b13fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b14074d80 0x7f6b14162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b12ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 0x7f6b14074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b13fff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b14074d80 0x7f6b14162a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:52346/0 (socket says 172.21.15.49:52346) 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b13fff640 1 -- 172.21.15.49:0/2301065698 learned_addr learned my addr 172.21.15.49:0/2301065698 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.096+0000 7f6b137fe640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b141592c0 0x7f6b1406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b13fff640 1 -- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b141592c0 msgr2=0x7f6b1406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b13fff640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b141592c0 0x7f6b1406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b13fff640 1 -- 172.21.15.49:0/2301065698 >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 msgr2=0x7f6b14074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b13fff640 1 --2- 172.21.15.49:0/2301065698 >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 0x7f6b14074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b13fff640 1 -- 172.21.15.49:0/2301065698 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b140ca960 con 0x7f6b14074d80 2024-05-09T08:57:44.098 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.097+0000 7f6b12ffd640 1 --2- 172.21.15.49:0/2301065698 >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 0x7f6b14074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.098+0000 7f6b13fff640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b14074d80 0x7f6b14162a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b1406f1f0 tx=0x7f6b0c02a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138f86c119720c81 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.098+0000 7f6b127fc640 1 -- 172.21.15.49:0/2301065698 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c036070 con 0x7f6b14074d80 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.098+0000 7f6b127fc640 1 -- 172.21.15.49:0/2301065698 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b0c031600 con 0x7f6b14074d80 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.098+0000 7f6b127fc640 1 -- 172.21.15.49:0/2301065698 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0319d0 con 0x7f6b14074d80 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 -- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b14074d80 msgr2=0x7f6b14162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b14074d80 0x7f6b14162a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b1406f1f0 tx=0x7f6b0c02a690 comp rx=0 tx=0).stop 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 -- 172.21.15.49:0/2301065698 shutdown_connections 2024-05-09T08:57:44.100 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b14074d80 0x7f6b14162a10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 --2- 172.21.15.49:0/2301065698 >> v2:172.21.15.49:3300/0 conn(0x7f6b140c8520 0x7f6b14074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 --2- 172.21.15.49:0/2301065698 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b141592c0 0x7f6b1406b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 -- 172.21.15.49:0/2301065698 >> 172.21.15.49:0/2301065698 conn(0x7f6b140ce5e0 msgr2=0x7f6b140ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 -- 172.21.15.49:0/2301065698 shutdown_connections 2024-05-09T08:57:44.101 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.099+0000 7f6b19f61640 1 -- 172.21.15.49:0/2301065698 wait complete. 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.101+0000 7f6b19f61640 1 Processor -- start 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.101+0000 7f6b19f61640 1 -- start start 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b19f61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b19f61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 0x7f6b140ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b19f61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141592c0 0x7f6b140cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59702/0 (socket says 172.21.15.49:59702) 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b137fe640 1 -- 172.21.15.49:0/1279035025 learned_addr learned my addr 172.21.15.49:0/1279035025 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b12ffd640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 0x7f6b140ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b13fff640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141592c0 0x7f6b140cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.103 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.103+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f6b141596a0 con 0x7f6b140c8520 2024-05-09T08:57:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1406b9d0 con 0x7f6b141592c0 2024-05-09T08:57:44.104 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1406cfc0 con 0x7f6b14074d80 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 -- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141592c0 msgr2=0x7f6b140cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141592c0 0x7f6b140cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 -- 172.21.15.49:0/1279035025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 msgr2=0x7f6b140ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 0x7f6b140ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141e94c0 con 0x7f6b14074d80 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b12ffd640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 0x7f6b140ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.104+0000 7f6b137fe640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400d5f0 tx=0x7f6b0400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.105+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b040125f0 con 0x7f6b14074d80 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.105+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b04019070 con 0x7f6b14074d80 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.105+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b040129d0 con 0x7f6b14074d80 2024-05-09T08:57:44.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.105+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141eb3a0 con 0x7f6b14074d80 2024-05-09T08:57:44.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.105+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141e9a40 con 0x7f6b14074d80 2024-05-09T08:57:44.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.107+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b140c6930 con 0x7f6b14074d80 2024-05-09T08:57:44.106 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.108+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b04012b70 con 0x7f6b14074d80 2024-05-09T08:57:44.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.108+0000 7f6afbfff640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 0x7f6b0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.107 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.108+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5857+0+0 (secure 0 0 0) 0x7f6b0408b1b0 con 0x7f6b14074d80 2024-05-09T08:57:44.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.109+0000 7f6b12ffd640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 0x7f6b0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.108 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.109+0000 7f6b12ffd640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 0x7f6b0006e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b140c7770 tx=0x7f6b08009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.113+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6b040593d0 con 0x7f6b14074d80 2024-05-09T08:57:44.140 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.140 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.149 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:44.195 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:44.196+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:44.261 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.261 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:44.295 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:44.296+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:44.382 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.382 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.392 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:44.396 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:44.397+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:44.497 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:44.498+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:44.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.500+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "3.1f", "id": [5, 6, 1, 7]} v 0) -- 0x7f6b14068c70 con 0x7f6b14074d80 2024-05-09T08:57:44.534 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.535 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.545 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:44.597 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:44.598+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:44.661 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.661 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.687+0000 7f6afbfff640 1 -- 172.21.15.49:0/1279035025 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "3.1f", "id": [5, 6, 1, 7]}]=0 set 3.1f pg_upmap_items mapping to [5->6,1->7] v51) ==== 148+0+0 (secure 0 0 0) 0x7f6b0405d6d0 con 0x7f6b14074d80 2024-05-09T08:57:44.687 INFO:teuthology.orchestra.run.smithi049.stderr:set 3.1f pg_upmap_items mapping to [5->6,1->7] 2024-05-09T08:57:44.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 msgr2=0x7f6b0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 0x7f6b0006e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b140c7770 tx=0x7f6b08009190 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 msgr2=0x7f6b140c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400d5f0 tx=0x7f6b0400dab0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 shutdown_connections 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f6b0006be60 0x7f6b0006e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141592c0 0x7f6b140cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6b140c8520 0x7f6b140ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 --2- 172.21.15.49:0/1279035025 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b14074d80 0x7f6b140c6da0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400d5f0 tx=0x7f6b0400dab0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 >> 172.21.15.49:0/1279035025 conn(0x7f6b140ce5e0 msgr2=0x7f6b14162860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 shutdown_connections 2024-05-09T08:57:44.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.692+0000 7f6b19f61640 1 -- 172.21.15.49:0/1279035025 wait complete. 2024-05-09T08:57:44.709 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:57:44.712 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:44.713+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:44.800 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:44.801+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:44.814 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.814 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.820+0000 7f8e663b0640 1 Processor -- start 2024-05-09T08:57:44.820 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.820+0000 7f8e663b0640 1 -- start start 2024-05-09T08:57:44.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.821+0000 7f8e663b0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e600cc8d0 0x7f8e6015b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.821+0000 7f8e663b0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 0x7f8e600d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.821+0000 7f8e663b0640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e600cce30 con 0x7f8e60155030 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.821+0000 7f8e663b0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8e600cb4f0 con 0x7f8e600cc8d0 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.821+0000 7f8e663b0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e600cd440 con 0x7f8e600c57f0 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8e60155030 0x7f8e600cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59628/0 (socket says 172.21.15.49:59628) 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e600cc8d0 0x7f8e6015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8e600cc8d0 0x7f8e6015b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42062/0 (socket says 172.21.15.49:42062) 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/540728793 learned_addr learned my addr 172.21.15.49:0/540728793 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:44.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e6494d640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 0x7f8e600d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/540728793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 msgr2=0x7f8e600d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 0x7f8e600d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/540728793 >> v1:172.21.15.49:6789/0 conn(0x7f8e60155030 legacy=0x7f8e600cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5effd640 1 -- 172.21.15.49:0/540728793 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696541695 0 0) 0x7f8e600cce30 con 0x7f8e60155030 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/540728793 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e600cb740 con 0x7f8e600cc8d0 2024-05-09T08:57:44.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.822+0000 7f8e6494d640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 0x7f8e600d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:44.825 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.823+0000 7f8e5f7fe640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600cc8d0 0x7f8e6015b850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009870 tx=0x7f8e5402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4b351a07ea4aea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.823+0000 7f8e5effd640 1 -- 172.21.15.49:0/540728793 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54037070 con 0x7f8e600cc8d0 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.823+0000 7f8e5effd640 1 -- 172.21.15.49:0/540728793 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e54032440 con 0x7f8e600cc8d0 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.823+0000 7f8e5effd640 1 -- 172.21.15.49:0/540728793 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54037070 con 0x7f8e600cc8d0 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 -- 172.21.15.49:0/540728793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600cc8d0 msgr2=0x7f8e6015b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600cc8d0 0x7f8e6015b850 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009870 tx=0x7f8e5402a250 comp rx=0 tx=0).stop 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 -- 172.21.15.49:0/540728793 shutdown_connections 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600c57f0 0x7f8e600d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 --2- 172.21.15.49:0/540728793 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600cc8d0 0x7f8e6015b850 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.828 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 -- 172.21.15.49:0/540728793 >> 172.21.15.49:0/540728793 conn(0x7f8e6006b780 msgr2=0x7f8e60072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 -- 172.21.15.49:0/540728793 shutdown_connections 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.824+0000 7f8e663b0640 1 -- 172.21.15.49:0/540728793 wait complete. 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.825+0000 7f8e663b0640 1 Processor -- start 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.825+0000 7f8e663b0640 1 -- start start 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 0x7f8e6015e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 0x7f8e601f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e60155410 con 0x7f8e600c57f0 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e600cba20 con 0x7f8e60155030 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e663b0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e60155590 con 0x7f8e600cc8d0 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e5ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42078/0 (socket says 172.21.15.49:42078) 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e5f7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 0x7f8e6015e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.829 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e6494d640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 0x7f8e601f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.826+0000 7f8e5ffff640 1 -- 172.21.15.49:0/165076675 learned_addr learned my addr 172.21.15.49:0/165076675 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 msgr2=0x7f8e601f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 0x7f8e601f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/165076675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 msgr2=0x7f8e6014d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 -- 172.21.15.49:0/165076675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6014bdf0 con 0x7f8e600cc8d0 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e6494d640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 0x7f8e601f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5ffff640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.827+0000 7f8e5f7fe640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 0x7f8e6015e070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e54031000 tx=0x7f8e540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.828+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54037040 con 0x7f8e600cc8d0 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.828+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e54004490 con 0x7f8e600cc8d0 2024-05-09T08:57:44.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.828+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6014c0d0 con 0x7f8e600cc8d0 2024-05-09T08:57:44.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.828+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6014c690 con 0x7f8e600cc8d0 2024-05-09T08:57:44.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.829+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e540325f0 con 0x7f8e600cc8d0 2024-05-09T08:57:44.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.829+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e40005290 con 0x7f8e600cc8d0 2024-05-09T08:57:44.831 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.829+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e54044020 con 0x7f8e600cc8d0 2024-05-09T08:57:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.830+0000 7f8e5cff9640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 0x7f8e4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.830+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 5886+0+0 (secure 0 0 0) 0x7f8e540aacc0 con 0x7f8e600cc8d0 2024-05-09T08:57:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.831+0000 7f8e5ffff640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 0x7f8e4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:44.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.831+0000 7f8e5ffff640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 0x7f8e4c06e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000f60 tx=0x7f8e50007e10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:44.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:44.834+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e540b1080 con 0x7f8e600cc8d0 2024-05-09T08:57:44.901 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:44.902+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:44.967 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:44.968 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:44.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:45.001 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:45.002+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:45.028 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.029+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f8e40005580 con 0x7f8e600cc8d0 2024-05-09T08:57:45.078 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:45.078 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:45.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:45.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:45.131 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:45.132+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:45.190 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:45.190 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:45.198 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:45.203 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:45.204+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:45.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.221+0000 7f7c5833b640 1 Processor -- start 2024-05-09T08:57:45.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.221+0000 7f7c5833b640 1 -- start start 2024-05-09T08:57:45.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c5833b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c500c6d50 0x7f7c50071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c5833b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c50072600 con 0x7f7c50154f70 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c5833b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c500728e0 con 0x7f7c50154b90 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c568d8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7c500c6d50 0x7f7c50071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c558d6640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7c50154f70 0x7f7c500c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59632/0 (socket says 172.21.15.49:59632) 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c558d6640 1 -- 172.21.15.49:0/1717572403 learned_addr learned my addr 172.21.15.49:0/1717572403 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.222+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7c500c5880 con 0x7f7c500c6d50 2024-05-09T08:57:45.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.223+0000 7f7c550d5640 1 -- 172.21.15.49:0/1717572403 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896557436 0 0) 0x7f7c50072600 con 0x7f7c50154f70 2024-05-09T08:57:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.223+0000 7f7c568d8640 1 -- 172.21.15.49:0/1717572403 >> v1:172.21.15.183:6789/0 conn(0x7f7c50154b90 legacy=0x7f7c500cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.223+0000 7f7c568d8640 1 -- 172.21.15.49:0/1717572403 >> v1:172.21.15.49:6789/0 conn(0x7f7c50154f70 legacy=0x7f7c500c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.223+0000 7f7c568d8640 1 -- 172.21.15.49:0/1717572403 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c500c5b60 con 0x7f7c500c6d50 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c568d8640 1 --2- 172.21.15.49:0/1717572403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500c6d50 0x7f7c50071820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009d90 tx=0x7f7c4802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25498b7ee9a4b79c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c550d5640 1 -- 172.21.15.49:0/1717572403 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48037070 con 0x7f7c500c6d50 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c550d5640 1 -- 172.21.15.49:0/1717572403 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c48007740 con 0x7f7c500c6d50 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c550d5640 1 -- 172.21.15.49:0/1717572403 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48007b10 con 0x7f7c500c6d50 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500c6d50 msgr2=0x7f7c50071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.224+0000 7f7c5833b640 1 --2- 172.21.15.49:0/1717572403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500c6d50 0x7f7c50071820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009d90 tx=0x7f7c4802a780 comp rx=0 tx=0).stop 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.225+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 shutdown_connections 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.225+0000 7f7c5833b640 1 --2- 172.21.15.49:0/1717572403 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c500c6d50 0x7f7c50071820 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.225+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 >> 172.21.15.49:0/1717572403 conn(0x7f7c50074840 msgr2=0x7f7c50074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.225+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 shutdown_connections 2024-05-09T08:57:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.225+0000 7f7c5833b640 1 -- 172.21.15.49:0/1717572403 wait complete. 2024-05-09T08:57:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.226+0000 7f7c5833b640 1 Processor -- start 2024-05-09T08:57:45.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.226+0000 7f7c5833b640 1 -- start start 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c500c6d50 0x7f7c500d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c50154b90 0x7f7c501f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c50154f70 0x7f7c50150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5015af40 con 0x7f7c50154b90 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c500c6120 con 0x7f7c500c6d50 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c5833b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7c500cb990 con 0x7f7c50154f70 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c560d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c500c6d50 0x7f7c500d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c560d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c500c6d50 0x7f7c500d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:52388/0 (socket says 172.21.15.49:52388) 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c560d7640 1 -- 172.21.15.49:0/3443252265 learned_addr learned my addr 172.21.15.49:0/3443252265 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:45.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.227+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c50154b90 0x7f7c501f8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c50154f70 0x7f7c50150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 -- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c50154f70 msgr2=0x7f7c50150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7c50154f70 0x7f7c50150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 -- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c50154b90 msgr2=0x7f7c501f8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c50154b90 0x7f7c501f8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5014afd0 con 0x7f7c500c6d50 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7c50154b90 0x7f7c501f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c500c6d50 0x7f7c500d0060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00c460 tx=0x7f7c4c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c016020 con 0x7f7c500c6d50 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c0052f0 con 0x7f7c500c6d50 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5014bdf0 con 0x7f7c500c6d50 2024-05-09T08:57:45.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.228+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c007260 con 0x7f7c500c6d50 2024-05-09T08:57:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.229+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c500d0be0 con 0x7f7c500c6d50 2024-05-09T08:57:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.229+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7c5004f660 con 0x7f7c500c6d50 2024-05-09T08:57:45.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.230+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c4c007400 con 0x7f7c500c6d50 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.230+0000 7f7c46ffd640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5886+0+0 (secure 0 0 0) 0x7f7c4c08a770 con 0x7f7c500c6d50 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v51) ==== 54+0+16 (secure 0 0 0) 0x7f7c4c020020 con 0x7f7c500c6d50 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7c50152670 0x7f7c50153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c50148b30 con 0x7f7c50152670 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7c50152670 0x7f7c50153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.231+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7c3406be30 0x7f7c3406e2f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7c500686f0 tx=0x7f7c40007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.232+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f7c50152670 0x7f7c50153540 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.232 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.232+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c50148b30 con 0x7f7c50152670 2024-05-09T08:57:45.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.247+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c5014aab0 con 0x7f7c50152670 2024-05-09T08:57:45.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.249+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c5014aab0 con 0x7f7c50152670 2024-05-09T08:57:45.248 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.249+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7c500d1100 0x7f7c50145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.249+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7c50145670 con 0x7f7c500d1100 2024-05-09T08:57:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.249+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7c500d1100 0x7f7c50145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.249 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.250+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f7c500d1100 0x7f7c50145130 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.251+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c50145670 con 0x7f7c500d1100 2024-05-09T08:57:45.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.280+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c50145670 con 0x7f7c500d1100 2024-05-09T08:57:45.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c50145670 con 0x7f7c500d1100 2024-05-09T08:57:45.282 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:45.283 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:45.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=52}) -- 0x7f7c50150c20 con 0x7f7c500c6d50 2024-05-09T08:57:45.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c50145670 con 0x7f7c500c6d50 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c091070 con 0x7f7c500c6d50 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7c5014f910 0x7f7c50061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.283+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f7c500ccf40 con 0x7f7c5014f910 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.284+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7c5014f910 0x7f7c50061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.284+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f7c5014f910 0x7f7c50061c70 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.285+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c500ccf40 con 0x7f7c5014f910 2024-05-09T08:57:45.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.299+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c50062480 con 0x7f7c5014f910 2024-05-09T08:57:45.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.302+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c50062480 con 0x7f7c5014f910 2024-05-09T08:57:45.301 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:45.302 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:45.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.302+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7c500cc2d0 0x7f7c500cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.302+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f7c50062400 con 0x7f7c500cc2d0 2024-05-09T08:57:45.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.302+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7c500cc2d0 0x7f7c500cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.303+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f7c500cc2d0 0x7f7c500cc730 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.303+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c50062400 con 0x7f7c500cc2d0 2024-05-09T08:57:45.303 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:45.304+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:45.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.318+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c50062400 con 0x7f7c500cc2d0 2024-05-09T08:57:45.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.320+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c50062400 con 0x7f7c500cc2d0 2024-05-09T08:57:45.319 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:45.320 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:45.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.320+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c500ccc70 0x7f7c5002c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.320+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f7c50062400 con 0x7f7c500ccc70 2024-05-09T08:57:45.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.320+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c500ccc70 0x7f7c5002c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.321+0000 7f7c560d7640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f7c500ccc70 0x7f7c5002c550 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.322+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c50062400 con 0x7f7c500ccc70 2024-05-09T08:57:45.342 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:45.343 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:45.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.351+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c50062400 con 0x7f7c500ccc70 2024-05-09T08:57:45.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:45.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.352+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c50062400 con 0x7f7c500ccc70 2024-05-09T08:57:45.352 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.353+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7c5002cfa0 0x7f7c5020b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.353+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f7c5002d420 con 0x7f7c5002cfa0 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.353+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7c5002cfa0 0x7f7c5020b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.353+0000 7f7c558d6640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f7c5002cfa0 0x7f7c5020b5a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.354+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c5002d420 con 0x7f7c5002cfa0 2024-05-09T08:57:45.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.383+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c5002d420 con 0x7f7c5002cfa0 2024-05-09T08:57:45.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.385+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c5002d420 con 0x7f7c5002cfa0 2024-05-09T08:57:45.384 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:45.385 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:45.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.385+0000 7f7c5833b640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7c5020bff0 0x7f7c5020e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:45.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.385+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f7c502111b0 con 0x7f7c5020bff0 2024-05-09T08:57:45.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.385+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7c5020bff0 0x7f7c5020e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:45.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.386+0000 7f7c568d8640 1 --2- 172.21.15.49:0/3443252265 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f7c5020bff0 0x7f7c5020e3e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:45.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.387+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c502111b0 con 0x7f7c5020bff0 2024-05-09T08:57:45.404 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:45.405+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:45.414 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.415+0000 7f7c5833b640 1 -- 172.21.15.49:0/3443252265 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7c5020e9a0 con 0x7f7c5020bff0 2024-05-09T08:57:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.417+0000 7f7c46ffd640 1 -- 172.21.15.49:0/3443252265 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7c5020e9a0 con 0x7f7c5020bff0 2024-05-09T08:57:45.416 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:45.416 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:45.421 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:45.421 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:45.493 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:45.493 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:45.493 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:45.502 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:45.505 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:45.506+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:45.604 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:45.604 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:45.604 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:45.604 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:45.604 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:45.605 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:45.607+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:45.613 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.689+0000 7f8e5cff9640 1 -- 172.21.15.49:0/165076675 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v52) ==== 97+0+0 (secure 0 0 0) 0x7f8e5404b020 con 0x7f8e600cc8d0 2024-05-09T08:57:45.688 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:57:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 msgr2=0x7f8e4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 0x7f8e4c06e320 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8e50000f60 tx=0x7f8e50007e10 comp rx=0 tx=0).stop 2024-05-09T08:57:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 msgr2=0x7f8e6015e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:45.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 0x7f8e6015e070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e54031000 tx=0x7f8e540046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 shutdown_connections 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e4c06be60 0x7f8e4c06e320 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e60155030 0x7f8e601f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e600cc8d0 0x7f8e6015e070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 --2- 172.21.15.49:0/165076675 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e600c57f0 0x7f8e6014d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 >> 172.21.15.49:0/165076675 conn(0x7f8e6006b780 msgr2=0x7f8e600c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 shutdown_connections 2024-05-09T08:57:45.692 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:45.692+0000 7f8e663b0640 1 -- 172.21.15.49:0/165076675 wait complete. 2024-05-09T08:57:45.706 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:45.707+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:45.762 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:45.762 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:45.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:45.762 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:45.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 99.081196167000002, 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:45.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.768 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 99.081193299999995, 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:45.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:45.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:45.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.776 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 99.081189655000003, 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.777 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:45.778 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:45.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:45.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:45.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:45.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:45.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:45.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:45.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:45.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:45.786 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.786 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.786 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.038380489999994, 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:45.787 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.788 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.788 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.788 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:45.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:45.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:45.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:45.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:45.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:45.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:45.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:45.798 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.033588500999997, 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:45.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:45.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:45.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.031082132999998, 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:45.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:45.809 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.810 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.810 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.810 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.029610743000006, 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:45.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:45.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.029534905000006, 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.815 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:45.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016713495000005, 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:45.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:45.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016692496999994, 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:45.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:45.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:45.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016688103000007, 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:45.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:45.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016683044999994, 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.838 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:45.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:45.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.847 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016679564, 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:45.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:45.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016674945999995, 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:45.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016671477000003, 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.862 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:45.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:45.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.867 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016668175999996, 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.868 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:45.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.874 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.016355106999995, 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:45.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:45.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:45.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.000279285000005, 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:45.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:45.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.000274606000005, 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.885 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:45.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:45.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.000268587999997, 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:45.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:45.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:45.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:45.897 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:45.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:45.899 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:45.899+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:46.000 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:46.001+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:46.043 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:46.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:46.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:46.043 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:46.051 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:46.136 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:46.137+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:46.194 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:46.194 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:46.195 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:46.195 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:46.195 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:46.201 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:46.202+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:46.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:46.302 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:46.303+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:46.347 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.683980084, 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.348 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.349 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:57:46.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.354 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.29132382, 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:57:46.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:57:46.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:57:46.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291276228, 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.361 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:57:46.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:57:46.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291226361, 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:57:46.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:57:46.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:57:46.374 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291220561, 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.375 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:46.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:57:46.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291108761, 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.381 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:57:46.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:57:46.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291011599, 2024-05-09T08:57:46.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:57:46.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.393 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.291005841, 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:57:46.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:57:46.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.290932201, 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:46.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:57:46.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:57:46.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.29088038899999, 2024-05-09T08:57:46.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:57:46.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.29087667, 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:57:46.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:57:46.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.417 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.805820213, 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:57:46.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.423 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.805803729, 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:57:46.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.429 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.805721772, 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:57:46.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:57:46.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.805674212, 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.437 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:57:46.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.602045767999996, 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.443 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:57:46.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:57:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.585966322999994, 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.449 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:57:46.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.585962007000006, 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:46.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:57:46.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:46.460 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.585953044000007, 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:57:46.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:57:46.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.585948983999998, 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:46.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:46.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:57:46.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:46.472 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:46.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:46.475 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:46.475+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:46.575 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:46.576+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:46.592 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:46.592 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:46.592 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:46.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:46.676 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:46.677+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:46.714 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:46.715 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:46.715 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:46.715 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:46.724 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:46.777 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:46.778+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.838 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.157867821, 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:46.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:46.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:46.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.157862252, 2024-05-09T08:57:46.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:46.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.15783056399999, 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:46.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.855 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.157825878, 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:46.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.15782204999999, 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:46.862 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:46.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.866 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.156939184, 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:46.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.871 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 95.111366298999997, 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:46.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 95.110693523999998, 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:46.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.107972856999993, 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:46.882 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.107964816000006, 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:46.888 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:46.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.107960496000004, 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.894 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:46.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:46.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.107956191, 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:46.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:46.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.107952943000001, 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:46.906 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:46.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:46.910 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.092880570000005, 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:46.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:46.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.092869323000002, 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:46.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.092864902000002, 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:46.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:46.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:46.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.928 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.076760937000003, 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:46.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:46.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.076749606000007, 2024-05-09T08:57:46.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:46.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:46.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:46.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.076741655999996, 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.940 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:46.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:46.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.945 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.076735778, 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:46.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:46.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:46.950 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:46.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:46.952 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:46.952+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:47.052 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:47.053+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:47.153 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:47.154+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:47.169 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:47.169 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:47.169 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:47.169 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:47.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:47.266 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:47.267+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:47.355 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:47.356+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:47.397 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:47.398 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:47.398 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:47.398 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:47.398 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:47.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:47.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:47.456 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:47.457+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:47.556 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:47.557+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.602+0000 7fd13387d640 1 Processor -- start 2024-05-09T08:57:47.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.602+0000 7fd13387d640 1 -- start start 2024-05-09T08:57:47.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.603+0000 7fd13387d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 0x7fd12c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.603+0000 7fd13387d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 0x7fd12c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.603+0000 7fd13387d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12c072600 con 0x7fd12c154f70 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.603+0000 7fd13387d640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd12c0728e0 con 0x7fd12c154b90 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.603+0000 7fd13387d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c0c5880 con 0x7fd12c0c6d50 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd130e18640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd12c154f70 0x7fd12c0c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59646/0 (socket says 172.21.15.49:59646) 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd130e18640 1 -- 172.21.15.49:0/2604672472 learned_addr learned my addr 172.21.15.49:0/2604672472 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131619640 1 --2- 172.21.15.49:0/2604672472 >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 0x7fd12c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131e1a640 1 --2- 172.21.15.49:0/2604672472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 0x7fd12c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd123fff640 1 -- 172.21.15.49:0/2604672472 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588759018 0 0) 0x7fd12c072600 con 0x7fd12c154f70 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd123fff640 1 -- 172.21.15.49:0/2604672472 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd118003520 con 0x7fd12c154f70 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131e1a640 1 -- 172.21.15.49:0/2604672472 >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 msgr2=0x7fd12c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131e1a640 1 --2- 172.21.15.49:0/2604672472 >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 0x7fd12c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131e1a640 1 -- 172.21.15.49:0/2604672472 >> v1:172.21.15.49:6789/0 conn(0x7fd12c154f70 legacy=0x7fd12c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131e1a640 1 -- 172.21.15.49:0/2604672472 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c0c5b60 con 0x7fd12c0c6d50 2024-05-09T08:57:47.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.604+0000 7fd131619640 1 --2- 172.21.15.49:0/2604672472 >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 0x7fd12c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.606+0000 7fd131e1a640 1 --2- 172.21.15.49:0/2604672472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 0x7fd12c0cb7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd124009d90 tx=0x7fd12402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91f755d3c26cac8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.606+0000 7fd123fff640 1 -- 172.21.15.49:0/2604672472 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124007740 con 0x7fd12c0c6d50 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.606+0000 7fd123fff640 1 -- 172.21.15.49:0/2604672472 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1240078e0 con 0x7fd12c0c6d50 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.606+0000 7fd123fff640 1 -- 172.21.15.49:0/2604672472 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124007cb0 con 0x7fd12c0c6d50 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 -- 172.21.15.49:0/2604672472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 msgr2=0x7fd12c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 --2- 172.21.15.49:0/2604672472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 0x7fd12c0cb7c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd124009d90 tx=0x7fd12402a6f0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 -- 172.21.15.49:0/2604672472 shutdown_connections 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 --2- 172.21.15.49:0/2604672472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c0c6d50 0x7fd12c0cb7c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 --2- 172.21.15.49:0/2604672472 >> v2:172.21.15.183:3300/0 conn(0x7fd12c154b90 0x7fd12c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 -- 172.21.15.49:0/2604672472 >> 172.21.15.49:0/2604672472 conn(0x7fd12c074840 msgr2=0x7fd12c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 -- 172.21.15.49:0/2604672472 shutdown_connections 2024-05-09T08:57:47.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.607+0000 7fd13387d640 1 -- 172.21.15.49:0/2604672472 wait complete. 2024-05-09T08:57:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.609+0000 7fd13387d640 1 Processor -- start 2024-05-09T08:57:47.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.609+0000 7fd13387d640 1 -- start start 2024-05-09T08:57:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12c0c6d50 0x7fd12c150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c154b90 0x7fd12c150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd12c154f70 0x7fd12c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c15af40 con 0x7fd12c0c6d50 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c0c6120 con 0x7fd12c154f70 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.610+0000 7fd13387d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c071820 con 0x7fd12c154b90 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c154b90 0x7fd12c150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c154b90 0x7fd12c150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59780/0 (socket says 172.21.15.49:59780) 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 -- 172.21.15.49:0/3684196379 learned_addr learned my addr 172.21.15.49:0/3684196379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd12c154f70 0x7fd12c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12c0c6d50 0x7fd12c150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 -- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd12c154f70 msgr2=0x7fd12c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd12c154f70 0x7fd12c0c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 -- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12c0c6d50 msgr2=0x7fd12c150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12c0c6d50 0x7fd12c150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c150ed0 con 0x7fd12c154b90 2024-05-09T08:57:47.614 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:47.614 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:47.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:47.614 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:47.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.9306622, 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:47.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:47.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:47.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:47.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:47.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.930637848, 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:47.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:47.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.930632093, 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 100.93061686199999, 2024-05-09T08:57:47.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:47.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:47.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:47.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:47.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 95.887278877, 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:47.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:47.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.647 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 95.887270916000006, 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:47.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:47.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:47.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.880062886999994, 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:47.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.880052141999997, 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:47.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:47.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866077568999998, 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:47.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866067459999996, 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:47.675 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:47.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:47.681 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866064821999998, 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.682 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:47.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866049219999994, 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:47.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:47.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:47.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866046570999998, 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:47.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:47.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:47.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.701 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866032445000002, 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.702 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:47.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:47.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.708 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866030166000002, 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:47.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:47.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:47.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.714 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866022260999998, 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:47.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.718 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 83.866020739999996, 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:47.719 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:47.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.845944891000002, 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:47.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.845923431000003, 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:47.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:47.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:47.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 77.845916426000002, 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:47.737 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:47.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:47.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:47.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:47.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:47.742 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:47.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:57:47.744 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:47.745+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd12c0c6d50 0x7fd12c150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd12c154b90 0x7fd12c150050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c390 tx=0x7fd11c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c017070 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.611+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c005ce0 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.612+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0072f0 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.612+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c1f0610 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.612+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c1f0bd0 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.614+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd12c04f660 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.615+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd11c01a070 con 0x7fd12c154b90 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.615+0000 7fd121ffb640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd11006be80 0x7fd11006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 5886+0+0 (secure 0 0 0) 0x7fd11c08a3a0 con 0x7fd12c154b90 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd11006be80 0x7fd11006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7fd11c08c980 con 0x7fd12c154b90 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd12c0c80d0 0x7fd12c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd12c074da0 con 0x7fd12c0c80d0 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.616+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd12c0c80d0 0x7fd12c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.617+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd11006be80 0x7fd11006e340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd128009ba0 tx=0x7fd128008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.617+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd12c0c80d0 0x7fd12c0cc770 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.618+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c074da0 con 0x7fd12c0c80d0 2024-05-09T08:57:47.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:47.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.635+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c0cccb0 con 0x7fd12c0c80d0 2024-05-09T08:57:47.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.638+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c0cccb0 con 0x7fd12c0c80d0 2024-05-09T08:57:47.749 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.638+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd12c1499f0 0x7fd12c1f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.638+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd12c0cffc0 con 0x7fd12c1499f0 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.639+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd12c1499f0 0x7fd12c1f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.639+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd12c1499f0 0x7fd12c1f3870 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.640+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c0cffc0 con 0x7fd12c1499f0 2024-05-09T08:57:47.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.671+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c150590 con 0x7fd12c1499f0 2024-05-09T08:57:47.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.672+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c150590 con 0x7fd12c1499f0 2024-05-09T08:57:47.751 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:47.752 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:47.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.672+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=53}) -- 0x7fd12c1f3db0 con 0x7fd12c154b90 2024-05-09T08:57:47.752 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.673+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd12c1f4180 con 0x7fd12c154b90 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.673+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd11c091070 con 0x7fd12c154b90 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.673+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd12c062180 0x7fd12c062590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.673+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fd12c062ad0 con 0x7fd12c062180 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.673+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd12c062180 0x7fd12c062590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.674+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd12c062180 0x7fd12c062590 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.675+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c062ad0 con 0x7fd12c062180 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.689+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c0ce660 con 0x7fd12c062180 2024-05-09T08:57:47.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.692+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c0ce660 con 0x7fd12c062180 2024-05-09T08:57:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.693+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd12c0cef30 0x7fd12c02c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.693+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fd12c2042f0 con 0x7fd12c0cef30 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.693+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd12c0cef30 0x7fd12c02c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.693+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd12c0cef30 0x7fd12c02c040 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.694+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c2042f0 con 0x7fd12c0cef30 2024-05-09T08:57:47.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.724+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c02c850 con 0x7fd12c0cef30 2024-05-09T08:57:47.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.726+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c02c850 con 0x7fd12c0cef30 2024-05-09T08:57:47.756 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:47.757 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:47.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.726+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd12c02cce0 0x7fd12c02d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.726+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fd12c207480 con 0x7fd12c02cce0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.727+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd12c02cce0 0x7fd12c02d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.727+0000 7fd131619640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd12c02cce0 0x7fd12c02d140 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.728+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c207480 con 0x7fd12c02cce0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.743+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c204bf0 con 0x7fd12c02cce0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.746+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c204bf0 con 0x7fd12c02cce0 2024-05-09T08:57:47.758 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:47.759 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.746+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd12c205100 0x7fd12c2074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.746+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fd12c20a2c0 con 0x7fd12c205100 2024-05-09T08:57:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.746+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd12c205100 0x7fd12c2074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.747+0000 7fd130e18640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd12c205100 0x7fd12c2074f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.760 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.748+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c20a2c0 con 0x7fd12c205100 2024-05-09T08:57:47.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.763+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c207a30 con 0x7fd12c205100 2024-05-09T08:57:47.765 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:47.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.765+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c207a30 con 0x7fd12c205100 2024-05-09T08:57:47.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.765+0000 7fd13387d640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd12c2080b0 0x7fd12c20a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.765+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fd12c20d270 con 0x7fd12c2080b0 2024-05-09T08:57:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.766+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd12c2080b0 0x7fd12c20a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.766+0000 7fd131e1a640 1 --2- 172.21.15.49:0/3684196379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd12c2080b0 0x7fd12c20a4a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.766 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.767+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd12c20d270 con 0x7fd12c2080b0 2024-05-09T08:57:47.799 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.799+0000 7fd13387d640 1 -- 172.21.15.49:0/3684196379 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd12c20a9e0 con 0x7fd12c2080b0 2024-05-09T08:57:47.802 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.803+0000 7fd121ffb640 1 -- 172.21.15.49:0/3684196379 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd12c20a9e0 con 0x7fd12c2080b0 2024-05-09T08:57:47.802 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:47.802 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:47.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:47.808 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:47.849 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:47.850+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.928+0000 7f8766dee640 1 Processor -- start 2024-05-09T08:57:47.927 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.928+0000 7f8766dee640 1 -- start start 2024-05-09T08:57:47.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.929+0000 7f8766dee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8760154b90 0x7f87600cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.929+0000 7f8766dee640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87600c6d50 0x7f87600cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f8766dee640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8760072600 con 0x7f8760154f70 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f8766dee640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f87600728e0 con 0x7f87600c6d50 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f8766dee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f87600c5880 con 0x7f8760154b90 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f876538b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f87600c6d50 0x7f87600cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f875ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8760154f70 0x7f87600c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59656/0 (socket says 172.21.15.49:59656) 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f875ffff640 1 -- 172.21.15.49:0/2963420671 learned_addr learned my addr 172.21.15.49:0/2963420671 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f8764b8a640 1 --2- 172.21.15.49:0/2963420671 >> v2:172.21.15.183:3300/0 conn(0x7f8760154b90 0x7f87600cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f875f7fe640 1 -- 172.21.15.49:0/2963420671 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94416676 0 0) 0x7f8760072600 con 0x7f8760154f70 2024-05-09T08:57:47.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f875f7fe640 1 -- 172.21.15.49:0/2963420671 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8750003520 con 0x7f8760154f70 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f876538b640 1 -- 172.21.15.49:0/2963420671 >> v2:172.21.15.183:3300/0 conn(0x7f8760154b90 msgr2=0x7f87600cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f876538b640 1 --2- 172.21.15.49:0/2963420671 >> v2:172.21.15.183:3300/0 conn(0x7f8760154b90 0x7f87600cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f876538b640 1 -- 172.21.15.49:0/2963420671 >> v1:172.21.15.49:6789/0 conn(0x7f8760154f70 legacy=0x7f87600c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.930+0000 7f876538b640 1 -- 172.21.15.49:0/2963420671 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87600c5b60 con 0x7f87600c6d50 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:47.931 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:47.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.931+0000 7f876538b640 1 --2- 172.21.15.49:0/2963420671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600c6d50 0x7f87600cb7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f87600cdf50 tx=0x7f875802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57c667c5e69fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.932+0000 7f875f7fe640 1 -- 172.21.15.49:0/2963420671 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758007740 con 0x7f87600c6d50 2024-05-09T08:57:47.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.932+0000 7f875f7fe640 1 -- 172.21.15.49:0/2963420671 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87580078e0 con 0x7f87600c6d50 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.932+0000 7f875f7fe640 1 -- 172.21.15.49:0/2963420671 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758007cb0 con 0x7f87600c6d50 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.932+0000 7f8766dee640 1 -- 172.21.15.49:0/2963420671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600c6d50 msgr2=0x7f87600cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.932+0000 7f8766dee640 1 --2- 172.21.15.49:0/2963420671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600c6d50 0x7f87600cb7c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f87600cdf50 tx=0x7f875802a710 comp rx=0 tx=0).stop 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 -- 172.21.15.49:0/2963420671 shutdown_connections 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 --2- 172.21.15.49:0/2963420671 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f87600c6d50 0x7f87600cb7c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 --2- 172.21.15.49:0/2963420671 >> v2:172.21.15.183:3300/0 conn(0x7f8760154b90 0x7f87600cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 -- 172.21.15.49:0/2963420671 >> 172.21.15.49:0/2963420671 conn(0x7f8760074840 msgr2=0x7f8760074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 -- 172.21.15.49:0/2963420671 shutdown_connections 2024-05-09T08:57:47.933 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.933+0000 7f8766dee640 1 -- 172.21.15.49:0/2963420671 wait complete. 2024-05-09T08:57:47.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.935+0000 7f8766dee640 1 Processor -- start 2024-05-09T08:57:47.934 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.935+0000 7f8766dee640 1 -- start start 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600c6d50 0x7f87601580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154f70 0x7f87601f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f876015af40 con 0x7f8760154b90 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87600c6120 con 0x7f87600c6d50 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f8766dee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8760071820 con 0x7f8760154f70 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f875ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f875ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42108/0 (socket says 172.21.15.49:42108) 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.936+0000 7f875ffff640 1 -- 172.21.15.49:0/760903574 learned_addr learned my addr 172.21.15.49:0/760903574 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f8764b8a640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600c6d50 0x7f87601580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.936 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f876538b640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154f70 0x7f87601f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 -- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154f70 msgr2=0x7f87601f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154f70 0x7f87601f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 -- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600c6d50 msgr2=0x7f87601580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600c6d50 0x7f87601580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 -- 172.21.15.49:0/760903574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87600d1230 con 0x7f8760154b90 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875ffff640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c410 tx=0x7f874c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c017070 con 0x7f8760154b90 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.937+0000 7f8766dee640 1 -- 172.21.15.49:0/760903574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8760155d40 con 0x7f8760154b90 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.938+0000 7f8766dee640 1 -- 172.21.15.49:0/760903574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8760156300 con 0x7f8760154b90 2024-05-09T08:57:47.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.938+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c005ce0 con 0x7f8760154b90 2024-05-09T08:57:47.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.939+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c0072f0 con 0x7f8760154b90 2024-05-09T08:57:47.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.939+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876004f660 con 0x7f8760154b90 2024-05-09T08:57:47.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.940+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f874c01a070 con 0x7f8760154b90 2024-05-09T08:57:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.941+0000 7f875d7fa640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 0x7f874406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.941+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5886+0+0 (secure 0 0 0) 0x7f874c08a2c0 con 0x7f8760154b90 2024-05-09T08:57:47.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.942+0000 7f8764b8a640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 0x7f874406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:47.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:47.942 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.942+0000 7f8764b8a640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 0x7f874406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8754009ad0 tx=0x7f8754008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:47.946 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:47.947+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:47.948 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:47.949+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f874c058590 con 0x7f8760154b90 2024-05-09T08:57:48.047 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:48.048+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:48.059 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.059 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:48.148 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:48.149+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:48.181 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.234 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:48.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.193+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f87600cbde0 con 0x7f8760154b90 2024-05-09T08:57:48.286 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:48.287+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:48.350 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:48.351+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:48.351 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.351 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.360 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:48.451 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:48.453+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:48.507 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.507 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.516 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:48.553 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:48.553+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:48.652 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:48.653+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:48.661 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.661 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.670 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.701+0000 7f875d7fa640 1 -- 172.21.15.49:0/760903574 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v53) ==== 91+0+0 (secure 0 0 0) 0x7f874c05c890 con 0x7f8760154b90 2024-05-09T08:57:48.704 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:57:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.704+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 msgr2=0x7f874406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.704+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 0x7f874406e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8754009ad0 tx=0x7f8754008040 comp rx=0 tx=0).stop 2024-05-09T08:57:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.704+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 msgr2=0x7f87601f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:48.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.704+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c410 tx=0x7f874c00c8d0 comp rx=0 tx=0).stop 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 shutdown_connections 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f874406be30 0x7f874406e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8760154f70 0x7f87601f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8760154b90 0x7f87601f3bf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 --2- 172.21.15.49:0/760903574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87600c6d50 0x7f87601580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:48.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 >> 172.21.15.49:0/760903574 conn(0x7f8760074840 msgr2=0x7f87600cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:48.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 shutdown_connections 2024-05-09T08:57:48.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:48.705+0000 7f874affd640 1 -- 172.21.15.49:0/760903574 wait complete. 2024-05-09T08:57:48.753 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:48.754+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:48.822 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.822 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.831 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:48.855 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:48.856+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:48.934 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:48.934 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:48.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:48.989 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:48.989+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:49.046 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.046 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:49.058 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:49.060+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:49.158 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:49.159+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:49.294 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:49.295+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:49.354 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.355 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.360 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:49.361+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:49.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:49.461 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:49.462+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:49.509 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.509 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:49.564 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:49.565+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:49.662 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.663 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.663 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:49.663+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:49.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:49.705 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:49.705 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:49.705 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-09T08:57:49.705 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.837173 2024-05-09T08:57:49.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.8371725123882788 2024-05-09T08:57:49.767 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:49.768+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:49.774 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.775 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:49.809 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:49.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.855+0000 7f3094258640 1 Processor -- start 2024-05-09T08:57:49.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.855+0000 7f3094258640 1 -- start start 2024-05-09T08:57:49.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3094258640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f308c0b3530 0x7f308c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3094258640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c0cbcd0 con 0x7f308c155190 2024-05-09T08:57:49.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3094258640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c15e410 con 0x7f308c0cb890 2024-05-09T08:57:49.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3094258640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f308c15ae60 con 0x7f308c0b3530 2024-05-09T08:57:49.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3091ff4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f308c155190 0x7f308c0cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59660/0 (socket says 172.21.15.49:59660) 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3091ff4640 1 -- 172.21.15.49:0/2189845827 learned_addr learned my addr 172.21.15.49:0/2189845827 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f30917f3640 1 --2- 172.21.15.49:0/2189845827 >> v2:172.21.15.183:3300/0 conn(0x7f308c0b3530 0x7f308c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3090ff2640 1 -- 172.21.15.49:0/2189845827 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350344795 0 0) 0x7f3088001110 con 0x7f308c155190 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.857+0000 7f3090ff2640 1 -- 172.21.15.49:0/2189845827 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3078003520 con 0x7f308c155190 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.858+0000 7f30917f3640 1 -- 172.21.15.49:0/2189845827 >> v1:172.21.15.183:6789/0 conn(0x7f308c0cb890 legacy=0x7f308c15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.858+0000 7f30917f3640 1 -- 172.21.15.49:0/2189845827 >> v1:172.21.15.49:6789/0 conn(0x7f308c155190 legacy=0x7f308c0cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.858+0000 7f30917f3640 1 -- 172.21.15.49:0/2189845827 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c15b120 con 0x7f308c0b3530 2024-05-09T08:57:49.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.859+0000 7f30917f3640 1 --2- 172.21.15.49:0/2189845827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0cb350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097a0 tx=0x7f307c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d99b530365c6bc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.859+0000 7f3090ff2640 1 -- 172.21.15.49:0/2189845827 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c037070 con 0x7f308c0b3530 2024-05-09T08:57:49.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.859+0000 7f3090ff2640 1 -- 172.21.15.49:0/2189845827 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c032640 con 0x7f308c0b3530 2024-05-09T08:57:49.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.859+0000 7f3090ff2640 1 -- 172.21.15.49:0/2189845827 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c032a10 con 0x7f308c0b3530 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 -- 172.21.15.49:0/2189845827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 msgr2=0x7f308c0cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 --2- 172.21.15.49:0/2189845827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0cb350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f307c0097a0 tx=0x7f307c02a2c0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 -- 172.21.15.49:0/2189845827 shutdown_connections 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 --2- 172.21.15.49:0/2189845827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0cb350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 -- 172.21.15.49:0/2189845827 >> 172.21.15.49:0/2189845827 conn(0x7f308c074a20 msgr2=0x7f308c074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 -- 172.21.15.49:0/2189845827 shutdown_connections 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.860+0000 7f3094258640 1 -- 172.21.15.49:0/2189845827 wait complete. 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.862+0000 7f3094258640 1 Processor -- start 2024-05-09T08:57:49.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.862+0000 7f3094258640 1 -- start start 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.863+0000 7f3094258640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3094258640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3094258640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59802/0 (socket says 172.21.15.49:59802) 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30927f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30927f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42124/0 (socket says 172.21.15.49:42124) 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3091ff4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3091ff4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:52436/0 (socket says 172.21.15.49:52436) 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3094258640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15b9c0 con 0x7f308c155190 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 -- 172.21.15.49:0/764773588 learned_addr learned my addr 172.21.15.49:0/764773588 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3094258640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f308c15b6e0 con 0x7f308c0b3530 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f3094258640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f308c15bca0 con 0x7f308c0cb890 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 -- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 msgr2=0x7f308c0b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.864+0000 7f30917f3640 1 -- 172.21.15.49:0/764773588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 msgr2=0x7f308c1f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30917f3640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30917f3640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c1f5590 con 0x7f308c0cb890 2024-05-09T08:57:49.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30927f5640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f3091ff4640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30917f3640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f307c035040 tx=0x7f307c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c037040 con 0x7f308c0cb890 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c032440 con 0x7f308c0cb890 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f3094258640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c1f4a10 con 0x7f308c0cb890 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c030990 con 0x7f308c0cb890 2024-05-09T08:57:49.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.865+0000 7f3094258640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1f4fd0 con 0x7f308c0cb890 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.868+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3068005290 con 0x7f308c0cb890 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.868+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f307c044020 con 0x7f308c0cb890 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.869+0000 7f30827fc640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.869+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 5886+0+0 (secure 0 0 0) 0x7f307c0ab080 con 0x7f308c0cb890 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.869+0000 7f3091ff4640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.870+0000 7f3091ff4640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 0x7f307006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30880013b0 tx=0x7f3088009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.870 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:49.870+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:49.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.873+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f307c079e20 con 0x7f308c0cb890 2024-05-09T08:57:49.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.923+0000 7fc90a9ce640 1 Processor -- start 2024-05-09T08:57:49.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.923+0000 7fc90a9ce640 1 -- start start 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc90a9ce640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc904154b90 0x7fc9040cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc90a9ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc904072600 con 0x7fc904154f70 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc9037fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fc904154f70 0x7fc9040c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59674/0 (socket says 172.21.15.49:59674) 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc9037fe640 1 -- 172.21.15.49:0/2476074287 learned_addr learned my addr 172.21.15.49:0/2476074287 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9040728e0 con 0x7fc9040c6d50 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc903fff640 1 --2- 172.21.15.49:0/2476074287 >> v2:172.21.15.183:3300/0 conn(0x7fc904154b90 0x7fc9040cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.928 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.924+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fc9040c5880 con 0x7fc904154b90 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3334625396 0 0) 0x7fc9040728e0 con 0x7fc9040c6d50 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8ec003520 con 0x7fc9040c6d50 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3534892862 0 0) 0x7fc904072600 con 0x7fc904154f70 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9040728e0 con 0x7fc904154f70 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 920444854 0 0) 0x7fc8ec003520 con 0x7fc9040c6d50 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 >> v2:172.21.15.183:3300/0 conn(0x7fc904154b90 msgr2=0x7fc9040cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 --2- 172.21.15.49:0/2476074287 >> v2:172.21.15.183:3300/0 conn(0x7fc904154b90 0x7fc9040cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 >> v1:172.21.15.49:6789/0 conn(0x7fc904154f70 legacy=0x7fc9040c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9040c5b60 con 0x7fc9040c6d50 2024-05-09T08:57:49.929 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fc8fc0019f0 con 0x7fc9040c6d50 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 >> v1:172.21.15.183:6789/0 conn(0x7fc9040c6d50 legacy=0x7fc904071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8ec003b40 0x7fc8ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec0079a0 0x7fc8ec009d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 0x7fc8ec00c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040c5880 con 0x7fc8ec003b40 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec00cfa0 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec00d1a0 con 0x7fc8ec0079a0 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 0x7fc8ec00c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.925+0000 7fc903fff640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8ec003b40 0x7fc8ec003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.930 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.926+0000 7fc908f6b640 1 -- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec0079a0 msgr2=0x7fc8ec009d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.926+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec0079a0 0x7fc8ec009d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.926+0000 7fc908f6b640 1 -- 172.21.15.49:0/2476074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8ec003b40 msgr2=0x7fc8ec003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.926+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8ec003b40 0x7fc8ec003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.926+0000 7fc908f6b640 1 -- 172.21.15.49:0/2476074287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9040c5b60 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc903fff640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc8ec003b40 0x7fc8ec003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc908f6b640 1 -- 172.21.15.49:0/2476074287 reap_dead start 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 0x7fc8ec00c760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c510 tx=0x7fc8ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601f4366c88899f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc040070 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc033e20 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.927+0000 7fc902ffd640 1 -- 172.21.15.49:0/2476074287 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009670 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.931 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 msgr2=0x7fc8ec00c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 0x7fc8ec00c760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00c510 tx=0x7fc8ec005830 comp rx=0 tx=0).stop 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 shutdown_connections 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2476074287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8ec00a2d0 0x7fc8ec00c760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 >> 172.21.15.49:0/2476074287 conn(0x7fc904074840 msgr2=0x7fc904074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 shutdown_connections 2024-05-09T08:57:49.932 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.928+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2476074287 wait complete. 2024-05-09T08:57:49.933 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:49.933 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.930+0000 7fc90a9ce640 1 Processor -- start 2024-05-09T08:57:49.937 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.930+0000 7fc90a9ce640 1 -- start start 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 0x7fc9041ef910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc904151be0 0x7fc9040b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9041511d0 0x7fc904144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc9037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc904151be0 0x7fc9040b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc9037fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc904151be0 0x7fc9040b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42134/0 (socket says 172.21.15.49:42134) 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc903fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 0x7fc9041ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc903fff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 0x7fc9041ef910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59812/0 (socket says 172.21.15.49:59812) 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc9037fe640 1 -- 172.21.15.49:0/2449540564 learned_addr learned my addr 172.21.15.49:0/2449540564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fc90415af40 con 0x7fc904151be0 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040c6120 con 0x7fc9041511d0 2024-05-09T08:57:49.938 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fc9040c6640 con 0x7fc8ec00a2d0 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.932+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9041511d0 0x7fc904144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 -- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 msgr2=0x7fc9041ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 0x7fc9041ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 -- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc904151be0 msgr2=0x7fc9040b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fc904151be0 0x7fc9040b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041efe50 con 0x7fc9041511d0 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8ec00a2d0 0x7fc9041ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:49.939 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9041511d0 0x7fc904144d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9040cdde0 tx=0x7fc8fc001cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc040040 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8fc04a070 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00c0e0 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90414f390 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.933+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9040d0150 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.934+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc90404f660 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.935+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8fc005320 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc900ff9640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc8f006be30 0x7fc8f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5886+0+0 (secure 0 0 0) 0x7fc8fc0b3720 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc8f006be30 0x7fc8f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7fc8fc0b9170 con 0x7fc9041511d0 2024-05-09T08:57:49.940 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc90414ee30 0x7fc90414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9040d0320 con 0x7fc90414ee30 2024-05-09T08:57:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.936+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc90414ee30 0x7fc90414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.937+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fc8f006be30 0x7fc8f006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009b40 tx=0x7fc8f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.937+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fc90414ee30 0x7fc90414cfb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.941 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.937+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9040d0320 con 0x7fc90414ee30 2024-05-09T08:57:49.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:49.955 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.956+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc9040cbde0 con 0x7fc90414ee30 2024-05-09T08:57:49.957 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.958+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc9040cbde0 con 0x7fc90414ee30 2024-05-09T08:57:49.958 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:49.960 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:49.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.958+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc9040b4ac0 0x7fc9040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.960 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.958+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc9040b53e0 con 0x7fc9040b4ac0 2024-05-09T08:57:49.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.958+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc9040b4ac0 0x7fc9040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.959+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fc9040b4ac0 0x7fc9040b4ea0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.960+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9040b53e0 con 0x7fc9040b4ac0 2024-05-09T08:57:49.970 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:49.971+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:49.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.978+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc90414e4f0 con 0x7fc9040b4ac0 2024-05-09T08:57:49.979 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.979+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc90414e4f0 con 0x7fc9040b4ac0 2024-05-09T08:57:49.979 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.980+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=54}) -- 0x7fc904075030 con 0x7fc9041511d0 2024-05-09T08:57:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.980+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc90414e4f0 con 0x7fc9041511d0 2024-05-09T08:57:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.981+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fc8fc0ba070 con 0x7fc9041511d0 2024-05-09T08:57:49.982 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:49.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.981+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc904062180 0x7fc9040625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:49.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.981+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fc904074d50 con 0x7fc904062180 2024-05-09T08:57:49.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.981+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc904062180 0x7fc9040625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:49.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.982+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fc904062180 0x7fc9040625d0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:49.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:49.982+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc904074d50 con 0x7fc904062180 2024-05-09T08:57:50.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.011+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc904074d50 con 0x7fc904062180 2024-05-09T08:57:50.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.014+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc904074d50 con 0x7fc904062180 2024-05-09T08:57:50.014 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:50.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:50.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.015+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc9041462b0 0x7fc9041466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.015+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fc904154140 con 0x7fc9041462b0 2024-05-09T08:57:50.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.015+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc9041462b0 0x7fc9041466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.015 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.016+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fc9041462b0 0x7fc9041466f0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.016 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.016+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc904154140 con 0x7fc9041462b0 2024-05-09T08:57:50.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.048+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc904154140 con 0x7fc9041462b0 2024-05-09T08:57:50.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.050+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc904154140 con 0x7fc9041462b0 2024-05-09T08:57:50.050 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:50.050 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:50.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.051+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc90402c550 0x7fc90402c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.051+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fc904150d40 con 0x7fc90402c550 2024-05-09T08:57:50.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.051+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc90402c550 0x7fc90402c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.052+0000 7fc903fff640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fc90402c550 0x7fc90402c950 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.052+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc904150d40 con 0x7fc90402c550 2024-05-09T08:57:50.073 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:50.074+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:50.086 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.086+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc904150d40 con 0x7fc90402c550 2024-05-09T08:57:50.087 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.088+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc904150d40 con 0x7fc90402c550 2024-05-09T08:57:50.088 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.089+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc904200c90 0x7fc904203080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.089+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fc90402d420 con 0x7fc904200c90 2024-05-09T08:57:50.088 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.089+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc904200c90 0x7fc904203080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.089+0000 7fc9037fe640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fc904200c90 0x7fc904203080 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.089 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.090+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc90402d420 con 0x7fc904200c90 2024-05-09T08:57:50.095 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:50.095 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:50.095 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:50.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:50.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.113+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc904073c20 con 0x7fc904200c90 2024-05-09T08:57:50.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.115+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc904073c20 con 0x7fc904200c90 2024-05-09T08:57:50.114 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.115+0000 7fc90a9ce640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc904203ad0 0x7fc904205ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.115+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fc904208de0 con 0x7fc904203ad0 2024-05-09T08:57:50.115 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.115+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc904203ad0 0x7fc904205ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.116+0000 7fc908f6b640 1 --2- 172.21.15.49:0/2449540564 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fc904203ad0 0x7fc904205ec0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.116 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.117+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc904208de0 con 0x7fc904203ad0 2024-05-09T08:57:50.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.142+0000 7fc90a9ce640 1 -- 172.21.15.49:0/2449540564 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc904207b40 con 0x7fc904203ad0 2024-05-09T08:57:50.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.144+0000 7fc900ff9640 1 -- 172.21.15.49:0/2449540564 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc904207b40 con 0x7fc904203ad0 2024-05-09T08:57:50.143 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:50.144 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:50.148 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:50.148 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:50.171 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:50.173+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:50.196 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.197+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 0.8371725123882788} v 0) -- 0x7f3068005850 con 0x7f308c0cb890 2024-05-09T08:57:50.234 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.234 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:50.234 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:50.234 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:50.235 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:50.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:50.292 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:50.293+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.361 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.680025138, 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:57:50.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:57:50.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.680019569, 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:57:50.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.372 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.679987881, 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:57:50.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:57:50.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.67998319500001, 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:57:50.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:57:50.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:57:50.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.679979367, 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:57:50.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.679096501, 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 98.633523616000005, 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:57:50.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:57:50.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 98.632850841000007, 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:57:50.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:57:50.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:57:50.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 92.630130174000001, 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.402 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:57:50.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.407 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 92.630122133, 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:57:50.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:57:50.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 92.630117812999998, 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:57:50.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:57:50.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.418 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 92.630113507999994, 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:57:50.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:57:50.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 92.630110259999995, 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:57:50.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:57:50.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.615037887, 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:57:50.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:57:50.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.615026639999996, 2024-05-09T08:57:50.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:50.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:57:50.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.438 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.615022218999997, 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:57:50.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.598918253999997, 2024-05-09T08:57:50.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:57:50.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:57:50.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:57:50.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:57:50.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.598906923000001, 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:57:50.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:57:50.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.598898973000004, 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.458 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:57:50.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.598893094999994, 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:50.467 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:50.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:50.469 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:50.470+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:50.570 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:50.571+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:50.627 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:50.627 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:50.627 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:50.637 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:50.671 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:50.672+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.704+0000 7f30827fc640 1 -- 172.21.15.49:0/764773588 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 0.8371725123882788}]=0 set osd.5 primary-affinity to 0.837173 (8548642) v54) ==== 155+0+0 (secure 0 0 0) 0x7f307c079720 con 0x7f308c0cb890 2024-05-09T08:57:50.704 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.5 primary-affinity to 0.837173 (8548642) 2024-05-09T08:57:50.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.711+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 msgr2=0x7f307006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.711+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 0x7f307006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f30880013b0 tx=0x7f3088009040 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.711+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 msgr2=0x7f308c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f307c035040 tx=0x7f307c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 shutdown_connections 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f307006be30 0x7f307006e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f308c155190 0x7f308c1f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f308c0cb890 0x7f308c15d4b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 --2- 172.21.15.49:0/764773588 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f308c0b3530 0x7f308c0b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 >> 172.21.15.49:0/764773588 conn(0x7f308c074a20 msgr2=0x7f308c074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 shutdown_connections 2024-05-09T08:57:50.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.712+0000 7f306ffff640 1 -- 172.21.15.49:0/764773588 wait complete. 2024-05-09T08:57:50.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:57:50.772 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:50.773+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:50.776 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:50.776 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:50.776 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:50.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:50.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.845+0000 7f5612ba2640 1 Processor -- start 2024-05-09T08:57:50.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.845+0000 7f5612ba2640 1 -- start start 2024-05-09T08:57:50.846 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.846+0000 7f5612ba2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.846+0000 7f5612ba2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f560c0cd6e0 0x7f560c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f5612ba2640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560c0c9be0 con 0x7f560c0cd280 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f5612ba2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f560c0cdc50 con 0x7f560c0cd6e0 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f5612ba2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f560c157ac0 con 0x7f560c151c00 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:59816/0 (socket says 172.21.15.49:59816) 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 -- 172.21.15.49:0/4213880931 learned_addr learned my addr 172.21.15.49:0/4213880931 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561113f640 1 --2- 172.21.15.49:0/4213880931 >> v2:172.21.15.49:3300/0 conn(0x7f560c0cd6e0 0x7f560c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 -- 172.21.15.49:0/4213880931 >> v2:172.21.15.49:3300/0 conn(0x7f560c0cd6e0 msgr2=0x7f560c15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 --2- 172.21.15.49:0/4213880931 >> v2:172.21.15.49:3300/0 conn(0x7f560c0cd6e0 0x7f560c15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 -- 172.21.15.49:0/4213880931 >> v1:172.21.15.183:6789/0 conn(0x7f560c0cd280 legacy=0x7f560c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.847+0000 7f561093e640 1 -- 172.21.15.49:0/4213880931 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c157da0 con 0x7f560c151c00 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.848+0000 7f561093e640 1 --2- 172.21.15.49:0/4213880931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f560000bb10 tx=0x7f560002c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589fb9745a139aae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.848+0000 7f560b7fe640 1 -- 172.21.15.49:0/4213880931 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600035070 con 0x7f560c151c00 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.848+0000 7f560b7fe640 1 -- 172.21.15.49:0/4213880931 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5600031610 con 0x7f560c151c00 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.848+0000 7f560b7fe640 1 -- 172.21.15.49:0/4213880931 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56000319e0 con 0x7f560c151c00 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 -- 172.21.15.49:0/4213880931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 msgr2=0x7f560c0c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 --2- 172.21.15.49:0/4213880931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f560000bb10 tx=0x7f560002c9b0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 -- 172.21.15.49:0/4213880931 shutdown_connections 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 --2- 172.21.15.49:0/4213880931 >> v2:172.21.15.49:3300/0 conn(0x7f560c0cd6e0 0x7f560c15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 --2- 172.21.15.49:0/4213880931 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c151c00 0x7f560c0c96a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 -- 172.21.15.49:0/4213880931 >> 172.21.15.49:0/4213880931 conn(0x7f560c06b200 msgr2=0x7f560c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 -- 172.21.15.49:0/4213880931 shutdown_connections 2024-05-09T08:57:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.849+0000 7f5612ba2640 1 -- 172.21.15.49:0/4213880931 wait complete. 2024-05-09T08:57:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.851+0000 7f5612ba2640 1 Processor -- start 2024-05-09T08:57:50.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.851+0000 7f5612ba2640 1 -- start start 2024-05-09T08:57:50.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c0cd6e0 0x7f560c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c151c00 0x7f560c1f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f560c158640 con 0x7f560c0cd280 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f561093e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f561093e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42152/0 (socket says 172.21.15.49:42152) 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f561093e640 1 -- 172.21.15.49:0/1057630799 learned_addr learned my addr 172.21.15.49:0/1057630799 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560c158360 con 0x7f560c151c00 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f560c151fe0 con 0x7f560c0cd6e0 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f560bfff640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c0cd6e0 0x7f560c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.852+0000 7f561113f640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c151c00 0x7f560c1f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 -- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c0cd6e0 msgr2=0x7f560c156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c0cd6e0 0x7f560c156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 -- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c151c00 msgr2=0x7f560c1f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c151c00 0x7f560c1f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c14bea0 con 0x7f560c0cd280 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f561093e640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5600000c00 tx=0x7f5600004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600035040 con 0x7f560c0cd280 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.853+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5600030bc0 con 0x7f560c0cd280 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.854+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560003a450 con 0x7f560c0cd280 2024-05-09T08:57:50.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.854+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c14c160 con 0x7f560c0cd280 2024-05-09T08:57:50.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.854+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c14c720 con 0x7f560c0cd280 2024-05-09T08:57:50.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.855+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c04f660 con 0x7f560c0cd280 2024-05-09T08:57:50.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.856+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5600038070 con 0x7f560c0cd280 2024-05-09T08:57:50.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.857+0000 7f56097fa640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 0x7f55f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:50.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.857+0000 7f560bfff640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 0x7f55f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.858+0000 7f560bfff640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 0x7f55f006e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f560c0cd1c0 tx=0x7f55f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:50.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.858+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5886+0+0 (secure 0 0 0) 0x7f56000a90e0 con 0x7f560c0cd280 2024-05-09T08:57:50.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:50.864+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5600078e00 con 0x7f560c0cd280 2024-05-09T08:57:50.873 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:50.874+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:50.975 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:50.976+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:50.992 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:50.992 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:50.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:50.992 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:50.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.309800645, 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:50.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:50.994 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:50.994 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:57:50.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:57:50.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:57:51.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:57:51.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:57:51.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:57:51.006 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:51.007 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:51.008 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.309776293, 2024-05-09T08:57:51.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:57:51.008 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.010 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:57:51.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:57:51.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:51.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:57:51.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:57:51.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:57:51.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:57:51.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:57:51.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.021 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.022 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:51.023 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:51.023 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.309770538, 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:51.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:57:51.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:57:51.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.309755307, 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:57:51.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.037 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.266417321999995, 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:57:51.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:57:51.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.266409361000001, 2024-05-09T08:57:51.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:51.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:57:51.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:57:51.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.259201332000004, 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:57:51.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:57:51.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.259190587000006, 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:51.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:57:51.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245216013999993, 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:51.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:57:51.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:51.066 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245205905000006, 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:57:51.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245203266999994, 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:57:51.072 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:57:51.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:57:51.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245187665000003, 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.078 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:57:51.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:57:51.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245185015999994, 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:51.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:57:51.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245170889999997, 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:51.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:57:51.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:51.095 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245168610999997, 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:51.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:57:51.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:57:51.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245160705999993, 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:57:51.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:51.105 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 87.245159185000006, 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:57:51.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:57:51.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.225083335999997, 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:51.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:57:51.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:51.115 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.225061875999998, 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:57:51.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.225054870999998, 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:57:51.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:57:51.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.125 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:51.127 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:51.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.115+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f560c0c8d10 con 0x7f560c0cd280 2024-05-09T08:57:51.127 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:51.127+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:51.228 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:51.229+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:51.280 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:51.281 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:57:51.281 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:51.290 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:51.328 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:51.329+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:51.430 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:51.431+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:51.445 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:51.445 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:51.445 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:51.445 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:51.445 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:51.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:51.531 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:51.532+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:51.567 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:51.567 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.886778593, 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.886758622, 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:51.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.885929856, 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:51.576 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:51.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:51.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:51.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:51.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:51.582 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.884926964, 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:51.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:51.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 104.884624133, 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:51.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.843437574999996, 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:51.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:51.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.601 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.843425549000003, 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:51.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:51.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.843412079000004, 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.842387677000005, 2024-05-09T08:57:51.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:51.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:51.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.841960388000004, 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.841844047999999, 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:51.623 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:51.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.840570052000004, 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:51.627 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:51.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.838820651999995, 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:51.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.838492508000002, 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:51.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.828363879999998, 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:51.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:51.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:51.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:51.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.649 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.837620878999999, 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:51.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:51.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:51.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.837611492999997, 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:51.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.837600449000007, 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:51.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:51.667 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.835865452999997, 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:51.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:51.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 81.805961246999999, 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:51.674 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:51.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:51.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:51.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:51.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:51.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:51.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:51.682 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:51.684 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:51.684 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:51.684+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:51.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.701+0000 7f56097fa640 1 -- 172.21.15.49:0/1057630799 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v55) ==== 101+0+0 (secure 0 0 0) 0x7f560007d100 con 0x7f560c0cd280 2024-05-09T08:57:51.701 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:57:51.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 msgr2=0x7f55f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:51.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 0x7f55f006e320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f560c0cd1c0 tx=0x7f55f8009040 comp rx=0 tx=0).stop 2024-05-09T08:57:51.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 msgr2=0x7f560c1f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:51.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5600000c00 tx=0x7f5600004730 comp rx=0 tx=0).stop 2024-05-09T08:57:51.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 shutdown_connections 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f55f006be60 0x7f55f006e320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c151c00 0x7f560c1f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f560c0cd6e0 0x7f560c156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 --2- 172.21.15.49:0/1057630799 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f560c0cd280 0x7f560c1f4920 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5600000c00 tx=0x7f5600004730 comp rx=0 tx=0).stop 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.704+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 >> 172.21.15.49:0/1057630799 conn(0x7f560c06b200 msgr2=0x7f560c0751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:51.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.706+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 shutdown_connections 2024-05-09T08:57:51.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:51.706+0000 7f5612ba2640 1 -- 172.21.15.49:0/1057630799 wait complete. 2024-05-09T08:57:51.785 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:51.786+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:51.802 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:51.803 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:51.803 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:51.803 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:51.811 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:51.896 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:51.897+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:51.955 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:51.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:51.956 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:51.956 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:51.956 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:51.965 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:52.006 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:52.007+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:52.088 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:52.089+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:52.109 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:52.109 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:52.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.42955381599999, 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:52.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:52.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.115 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.426134054, 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:52.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:52.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.425907545, 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:52.120 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:52.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:52.124 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.42585425199999, 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:52.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:52.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.377530680999996, 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:52.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.377004806000002, 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:52.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:52.137 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.376757474000001, 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:52.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.371318271000007, 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.142 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:52.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.363359106999994, 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:52.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:52.150 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.362794928, 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:52.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.362707854999996, 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.155 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:52.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:52.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.362638797000002, 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:52.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:52.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:52.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.361280488000006, 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:52.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.361235241000003, 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:52.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.360979974000003, 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:52.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:52.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 88.360941277999999, 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:52.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:52.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.183 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.347998219999994, 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:52.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.347359272000006, 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.188 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:52.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.346805328000002, 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.192 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:52.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:52.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 82.343951189999999, 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:52.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:52.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:52.200 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:52.201 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:52.201 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:52.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:52.203 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:52.203+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:52.303 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:52.305+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:52.346 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:52.346 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:52.346 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:52.346 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:52.355 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.358+0000 7fe6789ce640 1 Processor -- start 2024-05-09T08:57:52.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.358+0000 7fe6789ce640 1 -- start start 2024-05-09T08:57:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.359+0000 7fe6789ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe6740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe6789ce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6740c6d50 0x7fe6740cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe6789ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe674072600 con 0x7fe674154f70 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe6789ce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe6740728e0 con 0x7fe6740c6d50 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe6789ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe6740c5880 con 0x7fe674154b90 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6740c6d50 0x7fe6740cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe6740c6d50 0x7fe6740cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:42154/0 (socket says 172.21.15.49:42154) 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe67259c640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe6740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 -- 172.21.15.49:0/2290701340 learned_addr learned my addr 172.21.15.49:0/2290701340 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 -- 172.21.15.49:0/2290701340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 msgr2=0x7fe6740cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe6740cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 -- 172.21.15.49:0/2290701340 >> v1:172.21.15.49:6789/0 conn(0x7fe674154f70 legacy=0x7fe6740c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.360+0000 7fe672d9d640 1 -- 172.21.15.49:0/2290701340 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6740c5b60 con 0x7fe6740c6d50 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.361+0000 7fe672d9d640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6740c6d50 0x7fe6740cb7c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009e50 tx=0x7fe66c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36321e4a4ffd9bed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.361+0000 7fe67159a640 1 -- 172.21.15.49:0/2290701340 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c036070 con 0x7fe6740c6d50 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.362+0000 7fe67159a640 1 -- 172.21.15.49:0/2290701340 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66c031600 con 0x7fe6740c6d50 2024-05-09T08:57:52.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.362+0000 7fe67159a640 1 -- 172.21.15.49:0/2290701340 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0319d0 con 0x7fe6740c6d50 2024-05-09T08:57:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.362+0000 7fe6789ce640 1 -- 172.21.15.49:0/2290701340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6740c6d50 msgr2=0x7fe6740cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.362+0000 7fe6789ce640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6740c6d50 0x7fe6740cb7c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009e50 tx=0x7fe66c02a750 comp rx=0 tx=0).stop 2024-05-09T08:57:52.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 -- 172.21.15.49:0/2290701340 shutdown_connections 2024-05-09T08:57:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6740c6d50 0x7fe6740cb7c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 --2- 172.21.15.49:0/2290701340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe6740cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 -- 172.21.15.49:0/2290701340 >> 172.21.15.49:0/2290701340 conn(0x7fe674074840 msgr2=0x7fe674074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 -- 172.21.15.49:0/2290701340 shutdown_connections 2024-05-09T08:57:52.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.363+0000 7fe6789ce640 1 -- 172.21.15.49:0/2290701340 wait complete. 2024-05-09T08:57:52.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.364+0000 7fe6789ce640 1 Processor -- start 2024-05-09T08:57:52.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.365+0000 7fe6789ce640 1 -- start start 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6740c6d50 0x7fe67414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe67414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe674154f70 0x7fe6741f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe67415af40 con 0x7fe674154f70 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6740c6120 con 0x7fe6740c6d50 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe6789ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe674071820 con 0x7fe674154b90 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe67259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6740c6d50 0x7fe67414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe67259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6740c6d50 0x7fe67414b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:52488/0 (socket says 172.21.15.49:52488) 2024-05-09T08:57:52.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe67259c640 1 -- 172.21.15.49:0/1662274969 learned_addr learned my addr 172.21.15.49:0/1662274969 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe674154f70 0x7fe6741f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe67414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe67259c640 1 -- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 msgr2=0x7fe67414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.366+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe674154b90 0x7fe67414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe67259c640 1 -- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe674154f70 msgr2=0x7fe6741f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe674154f70 0x7fe6741f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe67259c640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67414d770 con 0x7fe6740c6d50 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe674154f70 0x7fe6741f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6740c6d50 0x7fe67414b830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe66400e9b0 tx=0x7fe66400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664017070 con 0x7fe6740c6d50 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66400c9a0 con 0x7fe6740c6d50 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cd90 con 0x7fe6740c6d50 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67414da50 con 0x7fe6740c6d50 2024-05-09T08:57:52.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.367+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67414b030 con 0x7fe6740c6d50 2024-05-09T08:57:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.369+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe67404f660 con 0x7fe6740c6d50 2024-05-09T08:57:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.369+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe66401a070 con 0x7fe6740c6d50 2024-05-09T08:57:52.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.370+0000 7fe662ffd640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe65406be60 0x7fe65406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.370+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5886+0+0 (secure 0 0 0) 0x7fe66408b220 con 0x7fe6740c6d50 2024-05-09T08:57:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.370+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v55) ==== 54+0+16 (secure 0 0 0) 0x7fe664059440 con 0x7fe6740c6d50 2024-05-09T08:57:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe65406be60 0x7fe65406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe6740d1180 0x7fe6741505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe67414d520 con 0x7fe6740d1180 2024-05-09T08:57:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe6740d1180 0x7fe6741505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe65406be60 0x7fe65406e320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe674153f40 tx=0x7fe668009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.371+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fe6740d1180 0x7fe6741505d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.373+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe67414d520 con 0x7fe6740d1180 2024-05-09T08:57:52.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.403+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe6740d0ac0 con 0x7fe6740d1180 2024-05-09T08:57:52.404 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:52.405+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:52.404 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.405+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe66c037040 con 0x7fe6740d1180 2024-05-09T08:57:52.404 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.405+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe6740b55c0 0x7fe6741f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.405+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe6741530d0 con 0x7fe6740b55c0 2024-05-09T08:57:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.406+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe6740b55c0 0x7fe6741f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.405 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.406+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe6740b55c0 0x7fe6741f45d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.406 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.407+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6741530d0 con 0x7fe6740b55c0 2024-05-09T08:57:52.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.437+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe674152df0 con 0x7fe6740b55c0 2024-05-09T08:57:52.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.439+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe674152df0 con 0x7fe6740b55c0 2024-05-09T08:57:52.439 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:52.439 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.440+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=56}) -- 0x7fe6741f4d60 con 0x7fe6740c6d50 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.440+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe674149a00 con 0x7fe6740c6d50 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.440+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe664021040 con 0x7fe6740c6d50 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.440+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe67414a4d0 0x7fe674061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.440+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fe6740ccf60 con 0x7fe67414a4d0 2024-05-09T08:57:52.440 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.441+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe67414a4d0 0x7fe674061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.441+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fe67414a4d0 0x7fe674061c70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.441 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.442+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6740ccf60 con 0x7fe67414a4d0 2024-05-09T08:57:52.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.472+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe674062480 con 0x7fe67414a4d0 2024-05-09T08:57:52.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.474+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe674062480 con 0x7fe67414a4d0 2024-05-09T08:57:52.473 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.474+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe6740cc2d0 0x7fe6740cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.474+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fe6740ccca0 con 0x7fe6740cc2d0 2024-05-09T08:57:52.474 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.475+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe6740cc2d0 0x7fe6740cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.475+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fe6740cc2d0 0x7fe6740cc760 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.475 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.476+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe66c045e50 con 0x7fe6740cc2d0 2024-05-09T08:57:52.498 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:52.498 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:52.498 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:52.498 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:52.498 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:52.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.505+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe67402c040 con 0x7fe6740cc2d0 2024-05-09T08:57:52.505 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:52.506+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:52.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.507+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe67402c040 con 0x7fe6740cc2d0 2024-05-09T08:57:52.507 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:52.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.507+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe67402c550 0x7fe67402c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.508+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fe67402ced0 con 0x7fe67402c550 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.508+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe67402c550 0x7fe67402c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.508+0000 7fe67259c640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fe67402c550 0x7fe67402c990 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.509+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe67402ced0 con 0x7fe67402c550 2024-05-09T08:57:52.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.538+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe67402ced0 con 0x7fe67402c550 2024-05-09T08:57:52.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.540+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe67402ced0 con 0x7fe67402c550 2024-05-09T08:57:52.539 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.540+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe674205650 0x7fe674207a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.540+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fe67402d550 con 0x7fe674205650 2024-05-09T08:57:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.540+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe674205650 0x7fe674207a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.541+0000 7fe671d9b640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fe674205650 0x7fe674207a40 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.542+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe67402d550 con 0x7fe674205650 2024-05-09T08:57:52.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.571+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe67402d550 con 0x7fe674205650 2024-05-09T08:57:52.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.574+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe67402d550 con 0x7fe674205650 2024-05-09T08:57:52.573 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:52.573 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.574+0000 7fe6789ce640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe674208490 0x7fe67420a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.574+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fe67420d790 con 0x7fe674208490 2024-05-09T08:57:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.574+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe674208490 0x7fe67420a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.575+0000 7fe672d9d640 1 --2- 172.21.15.49:0/1662274969 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fe674208490 0x7fe67420a880 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:52.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.575+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe67420d790 con 0x7fe674208490 2024-05-09T08:57:52.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.605+0000 7fe6789ce640 1 -- 172.21.15.49:0/1662274969 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe67420adc0 con 0x7fe674208490 2024-05-09T08:57:52.605 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:52.606+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:52.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:52.607+0000 7fe662ffd640 1 -- 172.21.15.49:0/1662274969 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe67420adc0 con 0x7fe674208490 2024-05-09T08:57:52.606 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:52.606 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:52.611 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:52.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:52.614 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:52.615 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:52.624 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:52.709 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:52.710+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:52.769 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:52.769 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:52.777 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:52.807 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:52.808+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:52.908 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:52.909+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:52.919 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:52.919 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:52.928 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:53.012 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:53.013+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:53.071 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.071 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:53.110 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:53.111+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:53.211 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:53.212+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:53.223 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.223 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.232 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:53.312 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:53.313+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:53.335 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.335 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.343 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:53.412 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:53.414+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:53.486 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.487 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:53.541 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:53.542+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:53.614 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:53.615+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:53.616 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.616 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:53.715 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:53.716+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:53.725 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-09T08:57:53.805 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.805 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.815 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:53.817 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:53.817+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:53.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.887+0000 7fe6abd5c640 1 Processor -- start 2024-05-09T08:57:53.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.887+0000 7fe6abd5c640 1 -- start start 2024-05-09T08:57:53.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.888+0000 7fe6abd5c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 0x7fe6a40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:53.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.888+0000 7fe6abd5c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a4072600 con 0x7fe6a4154f70 2024-05-09T08:57:53.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.888+0000 7fe6abd5c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6a40728e0 con 0x7fe6a40c6d50 2024-05-09T08:57:53.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.888+0000 7fe6abd5c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a40c5880 con 0x7fe6a4154b90 2024-05-09T08:57:53.888 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a9af8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 0x7fe6a40cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a92f7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe6a4154f70 0x7fe6a40c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49450/0 (socket says 172.21.15.49:49450) 2024-05-09T08:57:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a92f7640 1 -- 172.21.15.49:0/2444758528 learned_addr learned my addr 172.21.15.49:0/2444758528 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a9af8640 1 -- 172.21.15.49:0/2444758528 >> v1:172.21.15.183:6789/0 conn(0x7fe6a40c6d50 legacy=0x7fe6a4071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a9af8640 1 -- 172.21.15.49:0/2444758528 >> v1:172.21.15.49:6789/0 conn(0x7fe6a4154f70 legacy=0x7fe6a40c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:53.889 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.889+0000 7fe6a9af8640 1 -- 172.21.15.49:0/2444758528 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a40c5b60 con 0x7fe6a4154b90 2024-05-09T08:57:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.890+0000 7fe6a9af8640 1 --2- 172.21.15.49:0/2444758528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 0x7fe6a40cb280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00097d0 tx=0x7fe6a002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3926ec65740d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.890+0000 7fe6a8af6640 1 -- 172.21.15.49:0/2444758528 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0037070 con 0x7fe6a4154b90 2024-05-09T08:57:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.890+0000 7fe6a8af6640 1 -- 172.21.15.49:0/2444758528 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a0032440 con 0x7fe6a4154b90 2024-05-09T08:57:53.890 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.890+0000 7fe6a8af6640 1 -- 172.21.15.49:0/2444758528 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0032810 con 0x7fe6a4154b90 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 -- 172.21.15.49:0/2444758528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 msgr2=0x7fe6a40cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 --2- 172.21.15.49:0/2444758528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 0x7fe6a40cb280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00097d0 tx=0x7fe6a002a250 comp rx=0 tx=0).stop 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 -- 172.21.15.49:0/2444758528 shutdown_connections 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 --2- 172.21.15.49:0/2444758528 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a4154b90 0x7fe6a40cb280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 -- 172.21.15.49:0/2444758528 >> 172.21.15.49:0/2444758528 conn(0x7fe6a4074840 msgr2=0x7fe6a4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 -- 172.21.15.49:0/2444758528 shutdown_connections 2024-05-09T08:57:53.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.891+0000 7fe6abd5c640 1 -- 172.21.15.49:0/2444758528 wait complete. 2024-05-09T08:57:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.893+0000 7fe6abd5c640 1 Processor -- start 2024-05-09T08:57:53.892 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.893+0000 7fe6abd5c640 1 -- start start 2024-05-09T08:57:53.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:53.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a4154b90 0x7fe6a414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4154f70 0x7fe6a41f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a415af40 con 0x7fe6a4154f70 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a40c6120 con 0x7fe6a4154b90 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.894+0000 7fe6abd5c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a40cb7c0 con 0x7fe6a40c6d50 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6aa2f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4154f70 0x7fe6a41f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:38382/0 (socket says 172.21.15.49:38382) 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 -- 172.21.15.49:0/1417383564 learned_addr learned my addr 172.21.15.49:0/1417383564 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a92f7640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a4154b90 0x7fe6a414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 -- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a4154b90 msgr2=0x7fe6a414f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:53.895 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a4154b90 0x7fe6a414f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 -- 172.21.15.49:0/1417383564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4154f70 msgr2=0x7fe6a41f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4154f70 0x7fe6a41f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 -- 172.21.15.49:0/1417383564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a415aa90 con 0x7fe6a40c6d50 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe6a9af8640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098f0 tx=0x7fe6a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0037040 con 0x7fe6a40c6d50 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a0042070 con 0x7fe6a40c6d50 2024-05-09T08:57:53.896 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.895+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0030bd0 con 0x7fe6a40c6d50 2024-05-09T08:57:53.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.896+0000 7fe6abd5c640 1 -- 172.21.15.49:0/1417383564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a415ad30 con 0x7fe6a40c6d50 2024-05-09T08:57:53.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.896+0000 7fe6abd5c640 1 -- 172.21.15.49:0/1417383564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a40d1000 con 0x7fe6a40c6d50 2024-05-09T08:57:53.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.898+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe680005290 con 0x7fe6a40c6d50 2024-05-09T08:57:53.897 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.898+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6a0047460 con 0x7fe6a40c6d50 2024-05-09T08:57:53.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.899+0000 7fe69a7fc640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 0x7fe68806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:53.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.899+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 5886+0+0 (secure 0 0 0) 0x7fe6a00ab9d0 con 0x7fe6a40c6d50 2024-05-09T08:57:53.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.899+0000 7fe6a92f7640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 0x7fe68806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:53.898 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.900+0000 7fe6a92f7640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 0x7fe68806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41489b0 tx=0x7fe694007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:53.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:53.903+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe6a0032e20 con 0x7fe6a40c6d50 2024-05-09T08:57:53.919 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:53.920+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:53.958 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:53.958 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:53.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:54.020 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:54.021+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:54.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.113+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fe680005850 con 0x7fe6a40c6d50 2024-05-09T08:57:54.125 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:54.126+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:54.148 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:54.148 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:54.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:54.226 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:54.227+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:54.315 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:54.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:54.329 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:54.328+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:54.429 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:54.430+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:54.488 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:54.488 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:54.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:54.529 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:54.531+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:54.612 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:54.613 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:54.613 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:54.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:54.664 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:54.665+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:54.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.727+0000 7fe69a7fc640 1 -- 172.21.15.49:0/1417383564 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v56) ==== 87+0+0 (secure 0 0 0) 0x7fe6a0079c80 con 0x7fe6a40c6d50 2024-05-09T08:57:54.726 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 msgr2=0x7fe68806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 0x7fe68806e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6a41489b0 tx=0x7fe694007e80 comp rx=0 tx=0).stop 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 msgr2=0x7fe6a41f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00098f0 tx=0x7fe6a00046c0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 shutdown_connections 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe68806be30 0x7fe68806e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.729 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe6a4154f70 0x7fe6a41f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6a4154b90 0x7fe6a414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 --2- 172.21.15.49:0/1417383564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6a40c6d50 0x7fe6a41f5040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 >> 172.21.15.49:0/1417383564 conn(0x7fe6a4074840 msgr2=0x7fe6a414e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:54.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 shutdown_connections 2024-05-09T08:57:54.734 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.730+0000 7fe687fff640 1 -- 172.21.15.49:0/1417383564 wait complete. 2024-05-09T08:57:54.735 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:54.732+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.734+0000 7f5da61f9640 1 Processor -- start 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.734+0000 7f5da61f9640 1 -- start start 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5da61f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da00c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5da61f9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5da00733c0 0x7f5da00c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5da61f9640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da0070990 con 0x7f5da0152f30 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5da61f9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5da00c5fa0 con 0x7f5da00733c0 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5da61f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00ca5f0 con 0x7f5da00c9500 2024-05-09T08:57:54.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5da00733c0 0x7f5da00c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5da00733c0 0x7f5da00c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53780/0 (socket says 172.21.15.49:53780) 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 -- 172.21.15.49:0/1391290218 learned_addr learned my addr 172.21.15.49:0/1391290218 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da00c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 -- 172.21.15.49:0/1391290218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 msgr2=0x7f5da00c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da00c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 -- 172.21.15.49:0/1391290218 >> v1:172.21.15.183:6789/0 conn(0x7f5da0152f30 legacy=0x7f5da00c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.736+0000 7f5d9ffff640 1 -- 172.21.15.49:0/1391290218 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da015ccd0 con 0x7f5da00733c0 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.737+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da00c7b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009e50 tx=0x7f5d9802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad8579af75f0796 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.738+0000 7f5d9e7fc640 1 -- 172.21.15.49:0/1391290218 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98036070 con 0x7f5da00733c0 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.738+0000 7f5d9e7fc640 1 -- 172.21.15.49:0/1391290218 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98031600 con 0x7f5da00733c0 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.738+0000 7f5d9e7fc640 1 -- 172.21.15.49:0/1391290218 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980319f0 con 0x7f5da00733c0 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 -- 172.21.15.49:0/1391290218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 msgr2=0x7f5da00c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da00c7b40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009e50 tx=0x7f5d9802a7c0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 -- 172.21.15.49:0/1391290218 shutdown_connections 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da00c7b40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1391290218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da00c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 -- 172.21.15.49:0/1391290218 >> 172.21.15.49:0/1391290218 conn(0x7f5da00ce390 msgr2=0x7f5da00ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:54.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 -- 172.21.15.49:0/1391290218 shutdown_connections 2024-05-09T08:57:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.739+0000 7f5da61f9640 1 -- 172.21.15.49:0/1391290218 wait complete. 2024-05-09T08:57:54.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.741+0000 7f5da61f9640 1 Processor -- start 2024-05-09T08:57:54.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.741+0000 7f5da61f9640 1 -- start start 2024-05-09T08:57:54.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da01490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da01e4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5d9f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da01490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5d9f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da01490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53788/0 (socket says 172.21.15.49:53788) 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da0152f30 0x7f5da01e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5d9f7fe640 1 -- 172.21.15.49:0/1370610228 learned_addr learned my addr 172.21.15.49:0/1370610228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00caa80 con 0x7f5da00733c0 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00ca1a0 con 0x7f5da0152f30 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5da014f390 con 0x7f5da00c9500 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.743+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da0152f30 0x7f5da01e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da01e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 -- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 msgr2=0x7f5da01e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da01e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 -- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da0152f30 msgr2=0x7f5da01e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da0152f30 0x7f5da01e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da01495f0 con 0x7f5da00733c0 2024-05-09T08:57:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da00c9500 0x7f5da01e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:57:54.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.744+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5da00733c0 0x7f5da01490b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3f0 tx=0x7f5d9000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.745+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900052d0 con 0x7f5da00733c0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.745+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d90005470 con 0x7f5da00733c0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.745+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da014d270 con 0x7f5da00733c0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.745+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90007260 con 0x7f5da00733c0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.745+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da014f0b0 con 0x7f5da00733c0 2024-05-09T08:57:54.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.746+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5da0151530 con 0x7f5da00733c0 2024-05-09T08:57:54.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.747+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d90007400 con 0x7f5da00733c0 2024-05-09T08:57:54.748 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.748+0000 7f5d87fff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.748+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5886+0+0 (secure 0 0 0) 0x7f5d90056420 con 0x7f5da00733c0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.748+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7f5da0151530 con 0x7f5da00733c0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.748+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.748+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5da01f6a80 0x7f5da01f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.749+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5da004f660 con 0x7f5da01f6a80 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.749+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5da01f6a80 0x7f5da01f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.749+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f5da01f6a80 0x7f5da01f7040 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.749+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5d8c06be30 0x7f5d8c06e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5da006bbb0 tx=0x7f5d94008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.750+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da004f660 con 0x7f5da01f6a80 2024-05-09T08:57:54.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.766+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da0156430 con 0x7f5da01f6a80 2024-05-09T08:57:54.767 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.768+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da0156430 con 0x7f5da01f6a80 2024-05-09T08:57:54.767 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:54.768 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:54.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.768+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5da006b510 0x7f5da014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.768+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5da00731a0 con 0x7f5da006b510 2024-05-09T08:57:54.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.768+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5da006b510 0x7f5da014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.769+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f5da006b510 0x7f5da014dc20 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.769+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da00731a0 con 0x7f5da006b510 2024-05-09T08:57:54.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.785+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da014e430 con 0x7f5da006b510 2024-05-09T08:57:54.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.787+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da014e430 con 0x7f5da006b510 2024-05-09T08:57:54.786 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.787+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=57}) -- 0x7f5da00cef60 con 0x7f5da00733c0 2024-05-09T08:57:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.787+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5da00cec80 con 0x7f5da00733c0 2024-05-09T08:57:54.787 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.788+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f5d9008f060 con 0x7f5da00733c0 2024-05-09T08:57:54.788 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:54.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.788+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5da01f9350 0x7f5da0061c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.788+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f5da0070290 con 0x7f5da01f9350 2024-05-09T08:57:54.788 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.789+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5da01f9350 0x7f5da0061c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.789+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5da01f9350 0x7f5da0061c70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.789 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.790+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da0070290 con 0x7f5da01f9350 2024-05-09T08:57:54.804 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.805+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da00621b0 con 0x7f5da01f9350 2024-05-09T08:57:54.805 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.806+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da00621b0 con 0x7f5da01f9350 2024-05-09T08:57:54.806 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:54.806 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:54.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.807+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5da01e3330 0x7f5da01e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.807+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f5da0062c30 con 0x7f5da01e3330 2024-05-09T08:57:54.806 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.807+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5da01e3330 0x7f5da01e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.807+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f5da01e3330 0x7f5da01e3770 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.807 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.808+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d98037040 con 0x7f5da01e3330 2024-05-09T08:57:54.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.823+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da01e3f80 con 0x7f5da01e3330 2024-05-09T08:57:54.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.825+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da01e3f80 con 0x7f5da01e3330 2024-05-09T08:57:54.825 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:54.825 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:54.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.825+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5da002c550 0x7f5da002c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.825+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f5da01e40f0 con 0x7f5da002c550 2024-05-09T08:57:54.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.826+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5da002c550 0x7f5da002c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.826+0000 7f5d9f7fe640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f5da002c550 0x7f5da002c990 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.827+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da01e40f0 con 0x7f5da002c550 2024-05-09T08:57:54.831 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:54.832+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:54.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.842+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da01e40f0 con 0x7f5da002c550 2024-05-09T08:57:54.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.844+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da01e40f0 con 0x7f5da002c550 2024-05-09T08:57:54.843 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:54.843 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.844+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5da0209e00 0x7f5da020c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.844+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f5da01e4260 con 0x7f5da0209e00 2024-05-09T08:57:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.844+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5da0209e00 0x7f5da020c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.845+0000 7f5d9effd640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f5da0209e00 0x7f5da020c1f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.845+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da01e4260 con 0x7f5da0209e00 2024-05-09T08:57:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:57:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:57:54.846 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:57:54.846 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:54.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:54.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.861+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da01e4260 con 0x7f5da0209e00 2024-05-09T08:57:54.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.863+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da01e4260 con 0x7f5da0209e00 2024-05-09T08:57:54.863 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:54.863 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.863+0000 7f5da61f9640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5da020cc40 0x7f5da020f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.863+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f5da0211eb0 con 0x7f5da020cc40 2024-05-09T08:57:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.864+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5da020cc40 0x7f5da020f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.864+0000 7f5d9ffff640 1 --2- 172.21.15.49:0/1370610228 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f5da020cc40 0x7f5da020f030 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:54.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.865+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da0211eb0 con 0x7f5da020cc40 2024-05-09T08:57:54.881 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.882+0000 7f5da61f9640 1 -- 172.21.15.49:0/1370610228 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5da020f570 con 0x7f5da020cc40 2024-05-09T08:57:54.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:54.884+0000 7f5d87fff640 1 -- 172.21.15.49:0/1370610228 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5da020f570 con 0x7f5da020cc40 2024-05-09T08:57:54.884 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:54.884 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:54.888 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:54.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:54.933 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:54.933+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.981 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.300312998, 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037160890000000001, 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681070+0000", 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681075+0000", 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8250000000000004e-06 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:54.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681083+0000", 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0199999999999994e-06 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681120+0000", 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7421e-05 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681130+0000", 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0261000000000001e-05 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681311+0000", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018115300000000001 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682384+0000", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010728000000000001 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684786+0000", 2024-05-09T08:57:54.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024016089999999999 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.300293027, 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040701690000000002, 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:54.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:54.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681090+0000", 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7999999999999995e-07 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681093+0000", 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9330000000000001e-06 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681176+0000", 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3022999999999994e-05 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681185+0000", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5869999999999993e-06 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681325+0000", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014046800000000001 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681441+0000", 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011591699999999999 2024-05-09T08:57:54.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685160+0000", 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003718561 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:18 2.14 e20/17 2:2f55791f:::606.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.299464261, 2024-05-09T08:57:54.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023328559999999999, 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681918+0000", 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681920+0000", 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681928+0000", 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9759999999999995e-06 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:54.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681940+0000", 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2316e-05 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681949+0000", 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4169999999999998e-06 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682179+0000", 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229459 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682204+0000", 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5055e-05 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684206+0000", 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020017500000000001 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:54.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684251+0000", 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5577000000000001e-05 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:1 2.14 e20/17 2:292cf221:::200.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.29846136899999, 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0027676189999999998, 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:54.995 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682921+0000", 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682922+0000", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2799999999999996e-07 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682928+0000", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4219999999999997e-06 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682938+0000", 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0077e-05 2024-05-09T08:57:54.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682945+0000", 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8499999999999996e-06 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682999+0000", 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4098000000000003e-05 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:54.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683018+0000", 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8292999999999999e-05 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685121+0000", 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021032030000000001 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685689+0000", 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056814799999999998 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:54.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:3 2.14 e20/17 2:287034c4:::1.00000000.inode:head v 20'4, mlcod=0'0)", 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.298158538, 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024983750000000002, 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683224+0000", 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683225+0000", 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7100000000000001e-07 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683231+0000", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.314e-06 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683243+0000", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1405e-05 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683249+0000", 2024-05-09T08:57:55.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5980000000000002e-06 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683313+0000", 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4193e-05 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683333+0000", 2024-05-09T08:57:55.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9589000000000002e-05 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685693+0000", 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023595999999999999 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685723+0000", 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0005000000000002e-05 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.004 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:14 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.25697198, 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011285082, 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724411+0000", 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724413+0000", 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.04e-06 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724420+0000", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9469999999999997e-06 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724457+0000", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7700000000000002e-05 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724468+0000", 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0552e-05 2024-05-09T08:57:55.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724530+0000", 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2312999999999998e-05 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724637+0000", 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010710299999999999 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724759+0000", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000122033 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.733738+0000", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089784229999999993 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735603+0000", 2024-05-09T08:57:55.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001864987 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735652+0000", 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9571999999999999e-05 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735696+0000", 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3411999999999997e-05 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:17 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.256959954, 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011345297000000001, 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:55.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724423+0000", 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724434+0000", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1172000000000001e-05 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725165+0000", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073053899999999999 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725172+0000", 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2169999999999997e-06 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725224+0000", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1721999999999997e-05 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725344+0000", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012009400000000001 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735768+0000", 2024-05-09T08:57:55.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010424182000000001 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:18 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.256946484, 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011396675, 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724436+0000", 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724437+0000", 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5199999999999998e-07 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724438+0000", 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5209999999999999e-06 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725595+0000", 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011566650000000001 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725602+0000", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4619999999999999e-06 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725652+0000", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9938000000000002e-05 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725757+0000", 2024-05-09T08:57:55.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105078 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735833+0000", 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010075659000000001 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:9 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:55.019 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.255922082, 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089482799999999994, 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725461+0000", 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6499999999999998e-07 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725469+0000", 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6359999999999994e-06 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725481+0000", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1810999999999999e-05 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725500+0000", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8569000000000001e-05 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725568+0000", 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8110999999999999e-05 2024-05-09T08:57:55.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725591+0000", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3059000000000001e-05 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734366+0000", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087756199999999996 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.734409+0000", 2024-05-09T08:57:55.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2608999999999998e-05 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:7 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.255494793, 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096187619999999995, 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725888+0000", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725889+0000", 2024-05-09T08:57:55.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1900000000000002e-07 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725896+0000", 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9940000000000003e-06 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725906+0000", 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0156000000000001e-05 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725912+0000", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7009999999999999e-06 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726094+0000", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018129600000000001 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726112+0000", 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8417999999999999e-05 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735459+0000", 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.009346554 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735507+0000", 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8024e-05 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.1 epoch 20/17 rep_tid 11 entries 20'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.9:20 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.25537845300001, 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0095338119999999991, 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726004+0000", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726005+0000", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1099999999999995e-07 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726013+0000", 2024-05-09T08:57:55.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2160000000000006e-06 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726030+0000", 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.643e-05 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726043+0000", 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3370000000000001e-05 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726119+0000", 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6476000000000004e-05 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735538+0000", 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094187090000000008 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.032 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "pg_update_log_missing(2.12 epoch 20/17 rep_tid 3 entries 20'2 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.9:19 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.254104457, 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082460829999999995, 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727278+0000", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727279+0000", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000005e-07 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727289+0000", 2024-05-09T08:57:55.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7659999999999993e-06 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727313+0000", 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3811999999999998e-05 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727339+0000", 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6183e-05 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727416+0000", 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7336999999999995e-05 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735524+0000", 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081082979999999999 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:29 2.14 e20/17 2:2e2b5214:::610.00000000:head v 20'8, mlcod=20'5)", 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.252355057, 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065607119999999998, 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729028+0000", 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3499999999999995e-07 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729048+0000", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9675999999999999e-05 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729060+0000", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2074e-05 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729072+0000", 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1617000000000001e-05 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729131+0000", 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9040999999999999e-05 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729152+0000", 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0945e-05 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735556+0000", 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006404468 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735588+0000", 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2156000000000002e-05 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.041 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:27 2.12 e20/17 2:48cd7578:::60e.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.25202691299999, 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006210195, 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729356+0000", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0299999999999999e-07 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729361+0000", 2024-05-09T08:57:55.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1909999999999999e-06 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729371+0000", 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2080000000000006e-06 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729377+0000", 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729425+0000", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.871e-05 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.729444+0000", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8131999999999999e-05 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735527+0000", 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060833399999999996 2024-05-09T08:57:55.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.735566+0000", 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9115000000000001e-05 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.241898285, 2024-05-09T08:57:55.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098212100000000004, 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739484+0000", 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739486+0000", 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8360000000000001e-06 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739495+0000", 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7499999999999992e-06 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739602+0000", 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010720700000000001 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739611+0000", 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4619999999999996e-06 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739750+0000", 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013952300000000001 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739837+0000", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6730999999999993e-05 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.739985+0000", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014836499999999999 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.741382+0000", 2024-05-09T08:57:55.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013967859999999999 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749186+0000", 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007804203 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749253+0000", 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6925e-05 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.749306+0000", 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2422000000000001e-05 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.251155284000006, 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043251110000000004, 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730227+0000", 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730229+0000", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.341e-06 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730234+0000", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5140000000000001e-06 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730279+0000", 2024-05-09T08:57:55.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4409000000000001e-05 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730287+0000", 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3350000000000007e-06 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730348+0000", 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0520999999999999e-05 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730476+0000", 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012795800000000001 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730589+0000", 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011396999999999999 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732578+0000", 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019883629999999999 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734467+0000", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018887120000000001 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734514+0000", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7546999999999997e-05 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734553+0000", 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8441e-05 2024-05-09T08:57:55.057 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.251145898000004, 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053150879999999999, 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730237+0000", 2024-05-09T08:57:55.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730240+0000", 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.875e-06 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730311+0000", 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730316+0000", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0270000000000003e-06 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731016+0000", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00070024699999999998 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731098+0000", 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1808e-05 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731218+0000", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120025 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734092+0000", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028744640000000002 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735429+0000", 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013367660000000001 2024-05-09T08:57:55.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735500+0000", 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0557999999999998e-05 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735552+0000", 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2370999999999997e-05 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.251134854, 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044074300000000004, 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730248+0000", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8200000000000001e-07 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730259+0000", 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1211e-05 2024-05-09T08:57:55.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730351+0000", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1221e-05 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730358+0000", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3350000000000001e-06 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730421+0000", 2024-05-09T08:57:55.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459000000000003e-05 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730540+0000", 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118449 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734655+0000", 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041153730000000003 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.2.11:13 2.1 e20/17 2:81735765:::619.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.249399858000004, 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0021509250000000001, 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731983+0000", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731984+0000", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1500000000000003e-07 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731989+0000", 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5389999999999998e-06 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732004+0000", 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4596999999999999e-05 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732011+0000", 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5669999999999997e-06 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732064+0000", 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2703e-05 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732084+0000", 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9966000000000002e-05 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734094+0000", 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002009561 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734134+0000", 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0077000000000003e-05 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:24 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.219495652000006, 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059004890000000001, 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:57:55.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761888+0000", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3680000000000001e-06 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761900+0000", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1576000000000001e-05 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761921+0000", 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1135999999999999e-05 2024-05-09T08:57:55.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1440000000000003e-06 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762005+0000", 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5797999999999995e-05 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:57:55.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762087+0000", 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1858000000000001e-05 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762199+0000", 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111931 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763159+0000", 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096014899999999999 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767710+0000", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0045509620000000004 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767744+0000", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3988000000000001e-05 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767788+0000", 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3578999999999997e-05 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:57:55.078 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:57:55.078 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:55.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:55.080 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:55.080+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:55.180 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:55.181+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:55.195 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:55.195 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:55.195 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:55.195 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:55.206 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:55.290 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:55.291+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:55.350 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:55.350 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:55.351 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:55.351 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:55.358 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:55.382 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:55.383+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:55.482 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:55.483+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.823018909, 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062987599999999996, 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.503 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680255+0000", 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680264+0000", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680271+0000", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1489999999999997e-06 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680305+0000", 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4338000000000002e-05 2024-05-09T08:57:55.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680316+0000", 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0879999999999999e-05 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680443+0000", 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012703599999999999 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680611+0000", 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016818399999999999 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680762+0000", 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015036199999999999 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683410+0000", 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026476830000000001 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686439+0000", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030293120000000002 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686505+0000", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6549000000000003e-05 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686553+0000", 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8105000000000001e-05 2024-05-09T08:57:55.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:31 2.d e20/17 2:b50e409b:::mds_snaptable:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.81959914700001, 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028290670000000002, 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683674+0000", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683676+0000", 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0950000000000001e-06 2024-05-09T08:57:55.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683686+0000", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.027e-05 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683699+0000", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3376e-05 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683708+0000", 2024-05-09T08:57:55.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6279999999999994e-06 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683953+0000", 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024549700000000001 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683986+0000", 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2688999999999998e-05 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024548650000000001 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686504+0000", 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2646999999999998e-05 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.513 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.0.4:29 2.b e20/17 2:d0630e4c:::mds0_sessionmap:head v 20'1, mlcod=0'0)", 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.819372638, 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002607683, 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683901+0000", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683902+0000", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3510000000000001e-06 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683912+0000", 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.8039999999999996e-06 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683925+0000", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3081000000000001e-05 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683935+0000", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9869999999999994e-06 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683994+0000", 2024-05-09T08:57:55.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8455000000000002e-05 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684012+0000", 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8156e-05 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686465+0000", 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024535099999999999 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686509+0000", 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3339e-05 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.b epoch 20/17 rep_tid 122 entries 20'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.819319345, 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028876280000000002, 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.518 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683954+0000", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683955+0000", 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7999999999999996e-07 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683962+0000", 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5639999999999999e-06 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683977+0000", 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4280999999999999e-05 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683991+0000", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4034e-05 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684437+0000", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00044598800000000003 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686842+0000", 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002405281 2024-05-09T08:57:55.521 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:22 2.1a e20/17 2:5fd68f60:::61d.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.770995773999999, 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029159810000000002, 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732278+0000", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732279+0000", 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3099999999999997e-07 2024-05-09T08:57:55.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732286+0000", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0350000000000004e-06 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732295+0000", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1619999999999997e-06 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732302+0000", 2024-05-09T08:57:55.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2239999999999998e-06 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732354+0000", 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1881e-05 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732388+0000", 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4360000000000003e-05 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735135+0000", 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027463610000000001 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735194+0000", 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9227000000000001e-05 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.11 epoch 20/17 rep_tid 19 entries 20'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.11:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:55.526 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.770469899000005, 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027854199999999998, 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732804+0000", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732805+0000", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0240000000000001e-06 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732820+0000", 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5252999999999999e-05 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732846+0000", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6183e-05 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732864+0000", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7878000000000001e-05 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732955+0000", 2024-05-09T08:57:55.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1199000000000006e-05 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735589+0000", 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026338830000000001 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:8 2.13 e20/17 2:cdf3ac81:::616.00000000:head v 20'6, mlcod=20'3)", 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:55.530 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.770222567000005, 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025512400000000002, 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733051+0000", 2024-05-09T08:57:55.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733052+0000", 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.1100000000000005e-07 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733058+0000", 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4980000000000003e-06 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733069+0000", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0193999999999999e-05 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733075+0000", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9290000000000001e-06 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733154+0000", 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8071000000000005e-05 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733180+0000", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6412999999999999e-05 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735570+0000", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390143 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735602+0000", 2024-05-09T08:57:55.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2181000000000002e-05 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:34 2.11 e20/17 2:8975f766:::202.00000001:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.764783363999996, 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002076329, 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738490+0000", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738497+0000", 2024-05-09T08:57:55.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.092e-06 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738507+0000", 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.747e-06 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738520+0000", 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2774e-05 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738536+0000", 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6237999999999999e-05 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738600+0000", 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3566999999999996e-05 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.738623+0000", 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2903000000000001e-05 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740520+0000", 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001897391 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.740567+0000", 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6616999999999998e-05 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.539 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 15 entries 20'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.13:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.756824199999997, 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020586100000000002, 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746449+0000", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746451+0000", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.717e-06 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746467+0000", 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.5874000000000001e-05 2024-05-09T08:57:55.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746503+0000", 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6216e-05 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746523+0000", 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0234e-05 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746685+0000", 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016177499999999999 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748508+0000", 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001822794 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:24 2.13 e20/17 2:cbf94b35:::mds3_inotable:head v 20'7, mlcod=20'5)", 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:55.543 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.756260021000003, 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00385733, 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747014+0000", 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747015+0000", 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2500000000000001e-06 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747026+0000", 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1267000000000001e-05 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747042+0000", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6098000000000001e-05 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747052+0000", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3009999999999995e-06 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747833+0000", 2024-05-09T08:57:55.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00078122399999999996 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747852+0000", 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8941999999999999e-05 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750819+0000", 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029672190000000001 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750871+0000", 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2028999999999999e-05 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:3 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.756172948, 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037639240000000001, 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.548 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747101+0000", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8899999999999998e-07 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747108+0000", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8719999999999996e-06 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747117+0000", 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7320000000000004e-06 2024-05-09T08:57:55.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747124+0000", 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.426e-06 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747598+0000", 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000474539 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747619+0000", 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1022e-05 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750810+0000", 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00319068 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750865+0000", 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4864000000000002e-05 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:19 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.756103890000006, 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037862579999999998, 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747170+0000", 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-09T08:57:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747183+0000", 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2361000000000001e-05 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748262+0000", 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010796390000000001 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748272+0000", 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.006e-05 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748335+0000", 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2624000000000002e-05 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748354+0000", 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9286000000000001e-05 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750907+0000", 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002552693 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750956+0000", 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9063000000000001e-05 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.557 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:4 2.1a e20/17 2:5fdc322d:::61e.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.754745580999995, 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030836570000000001, 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748528+0000", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748529+0000", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4099999999999998e-07 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748536+0000", 2024-05-09T08:57:55.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8050000000000007e-06 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748545+0000", 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2709999999999992e-06 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748551+0000", 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.5719999999999999e-06 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9169000000000003e-05 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748619+0000", 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8457999999999998e-05 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751578+0000", 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002959293 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751612+0000", 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3448000000000002e-05 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:20 2.11 e20/17 2:890f50f7:::626.00000000:head v 20'5, mlcod=20'2)", 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.754700334000006, 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030343219999999999, 2024-05-09T08:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748573+0000", 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748574+0000", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0100000000000005e-07 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748594+0000", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0451000000000001e-05 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748601+0000", 2024-05-09T08:57:55.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8689999999999998e-06 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748612+0000", 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0689999999999999e-05 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748653+0000", 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.108e-05 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7836e-05 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751538+0000", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002866937 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751608+0000", 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9957999999999997e-05 2024-05-09T08:57:55.566 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:13 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'6, mlcod=20'2)", 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.754445067000006, 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028097729999999998, 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748829+0000", 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8299999999999996e-07 2024-05-09T08:57:55.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748836+0000", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6460000000000004e-06 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748845+0000", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4490000000000005e-06 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748852+0000", 2024-05-09T08:57:55.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3180000000000003e-06 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748999+0000", 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000146935 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749016+0000", 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7802e-05 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751614+0000", 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025973200000000002 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751638+0000", 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4519999999999999e-05 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:14 2.1a e20/17 2:594e21d0:::623.00000000:head v 20'7, mlcod=20'2)", 2024-05-09T08:57:55.571 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 91.754406371000002, 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028062479999999999, 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748867+0000", 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748868+0000", 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9200000000000001e-07 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748872+0000", 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6489999999999999e-06 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748889+0000", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6957e-05 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748908+0000", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8969000000000001e-05 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749190+0000", 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00028154500000000002 2024-05-09T08:57:55.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749207+0000", 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6937999999999999e-05 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751641+0000", 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024343500000000001 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751674+0000", 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2348000000000003e-05 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:13 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'8, mlcod=20'6)", 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.741463312999997, 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039472739999999997, 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.576 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761810+0000", 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761812+0000", 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.584e-06 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761825+0000", 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3464e-05 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761847+0000", 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.124e-05 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761857+0000", 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0268999999999999e-05 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762749+0000", 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00089236100000000004 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762771+0000", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1999000000000001e-05 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765703+0000", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029319849999999998 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765758+0000", 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4372000000000001e-05 2024-05-09T08:57:55.580 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.18 epoch 20/17 rep_tid 17 entries 20'3 (0'0) error 2:1f0916d5:::204.00000003:head by mds.4.15:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.740824364999995, 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003387401, 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.581 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762449+0000", 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762451+0000", 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3859999999999999e-06 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762469+0000", 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8087000000000001e-05 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762493+0000", 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4615000000000001e-05 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762510+0000", 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6364000000000001e-05 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762641+0000", 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013114999999999999 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765837+0000", 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003195799 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:14 2.13 e20/17 2:c81eaac8:::62d.00000000:head v 20'9, mlcod=20'6)", 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.740270421000005, 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037214969999999998, 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.585 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763003+0000", 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763004+0000", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6700000000000003e-07 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763014+0000", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0473e-05 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763025+0000", 2024-05-09T08:57:55.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0181e-05 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763031+0000", 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6599999999999998e-06 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763081+0000", 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9610000000000001e-05 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:55.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763099+0000", 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8175000000000001e-05 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766668+0000", 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003569259 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766725+0000", 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6471999999999997e-05 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "pg_update_log_missing(2.d epoch 20/17 rep_tid 24 entries 20'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.15:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 20'1)", 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.737416283000002, 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029417950000000001, 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:55.590 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "reached pg", 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765857+0000", 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765858+0000", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.029e-06 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765869+0000", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0847e-05 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765896+0000", 2024-05-09T08:57:55.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6704e-05 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765914+0000", 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7876999999999998e-05 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765998+0000", 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4258999999999994e-05 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:55.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768799+0000", 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028010790000000002 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:55.594 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:55.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:55.596 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:55.597+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:55.696 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:55.698+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:55.730 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:57:55.730 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:57:55.730 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-09T08:57:55.730 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-09T08:57:55.730 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-09T08:57:55.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-09T08:57:55.739 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:55.740 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:55.740 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:55.740 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:55.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:55.834 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:55.835+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:55.844 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:57:55.844 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-09T08:57:55.845 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:57:55.893 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:55.893 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:55.893 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:55.898 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:55.900+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:55.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:55.999 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:56.000+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:7 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.047 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.322274595, 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010924086, 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724842+0000", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724844+0000", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.761e-06 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724851+0000", 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8169999999999999e-06 2024-05-09T08:57:56.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724911+0000", 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9293999999999998e-05 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724920+0000", 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4660000000000005e-06 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724993+0000", 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3184999999999998e-05 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725095+0000", 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000102051 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725226+0000", 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013086400000000001 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727821+0000", 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025946390000000001 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735666+0000", 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0078450459999999996 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735722+0000", 2024-05-09T08:57:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6416999999999999e-05 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735766+0000", 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3545999999999999e-05 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:15 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.322253996, 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0074406389999999998, 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724862+0000", 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724863+0000", 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.79e-07 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724866+0000", 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4039999999999999e-06 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725533+0000", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000667003 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725541+0000", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5970000000000003e-06 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725609+0000", 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8231000000000005e-05 2024-05-09T08:57:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725715+0000", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000106327 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.725810+0000", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5117000000000004e-05 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.727809+0000", 2024-05-09T08:57:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019989090000000001 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732235+0000", 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044256149999999999 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732270+0000", 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5478999999999997e-05 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732303+0000", 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2678000000000001e-05 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:19 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.322249125, 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011046926, 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.060 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724867+0000", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724868+0000", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2000000000000001e-07 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724886+0000", 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7991000000000001e-05 2024-05-09T08:57:56.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726137+0000", 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012516039999999999 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726149+0000", 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1731e-05 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726218+0000", 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.923e-05 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.726334+0000", 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115957 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735914+0000", 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095800929999999996 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:27 2.12 2:48cd7578:::60e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.31869082599999, 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075669700000000001, 2024-05-09T08:57:56.065 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728426+0000", 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728427+0000", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0380000000000001e-06 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728431+0000", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.826e-06 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728460+0000", 2024-05-09T08:57:56.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8515000000000001e-05 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728468+0000", 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0879999999999995e-06 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728517+0000", 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9014000000000002e-05 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728601+0000", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4097000000000005e-05 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728736+0000", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013478399999999999 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732461+0000", 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037245469999999999 2024-05-09T08:57:56.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735942+0000", 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0034816230000000001 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735970+0000", 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7424999999999999e-05 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735993+0000", 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3013000000000001e-05 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:28 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.318682598, 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042667759999999999, 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.071 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728434+0000", 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728437+0000", 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6410000000000002e-06 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728494+0000", 2024-05-09T08:57:56.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7392999999999998e-05 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728502+0000", 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5129999999999999e-06 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728559+0000", 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7380000000000003e-05 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728634+0000", 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4998000000000003e-05 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728752+0000", 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118059 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729437+0000", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068526000000000004 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732631+0000", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031937670000000001 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732660+0000", 2024-05-09T08:57:56.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9268000000000002e-05 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732701+0000", 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0263999999999999e-05 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:33 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:56.078 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.318678789, 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052209910000000003, 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:56.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728438+0000", 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.05e-07 2024-05-09T08:57:56.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728443+0000", 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.609e-06 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728534+0000", 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.0505000000000002e-05 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728542+0000", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8350000000000004e-06 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728599+0000", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7496e-05 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728727+0000", 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012766700000000001 2024-05-09T08:57:56.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728923+0000", 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019611100000000001 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.732499+0000", 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003575651 2024-05-09T08:57:56.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733567+0000", 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00106842 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733619+0000", 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.185e-05 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.733659+0000", 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9641999999999998e-05 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:25 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.316122761000003, 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0044948080000000003, 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:56.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730994+0000", 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730996+0000", 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9709999999999998e-06 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731004+0000", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0260000000000007e-06 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731030+0000", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6234e-05 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731039+0000", 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.5440000000000004e-06 2024-05-09T08:57:56.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731115+0000", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5150999999999999e-05 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731276+0000", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00016165900000000001 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731405+0000", 2024-05-09T08:57:56.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128374 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.733795+0000", 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002390498 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735333+0000", 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00153802 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735439+0000", 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010629900000000001 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.735489+0000", 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9032000000000001e-05 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.091 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:9 2.3 2:c04be1b2:::621.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.301621953999998, 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040048150000000001, 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745494+0000", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745497+0000", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2500000000000001e-06 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745505+0000", 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5380000000000002e-06 2024-05-09T08:57:56.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745547+0000", 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1686e-05 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745558+0000", 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1223000000000001e-05 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745660+0000", 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000101319 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746657+0000", 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00099699399999999996 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000151846 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748671+0000", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018627400000000001 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749372+0000", 2024-05-09T08:57:56.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070125299999999999 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749443+0000", 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0766999999999995e-05 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749499+0000", 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6199000000000002e-05 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.301606483, 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052604110000000004, 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.098 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:56.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745510+0000", 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.39e-07 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745513+0000", 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.898e-06 2024-05-09T08:57:56.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746498+0000", 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098524700000000008 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746507+0000", 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8799999999999997e-06 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748086+0000", 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015782789999999999 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748165+0000", 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.9354e-05 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748310+0000", 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014459100000000001 2024-05-09T08:57:56.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749690+0000", 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013799439999999999 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750664+0000", 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097453600000000002 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750726+0000", 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2253000000000002e-05 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750770+0000", 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.409e-05 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.104 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.301601739000006, 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066389439999999999, 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745515+0000", 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9499999999999998e-07 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745520+0000", 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8960000000000002e-06 2024-05-09T08:57:56.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748651+0000", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031315610000000002 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748663+0000", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1722000000000001e-05 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748724+0000", 2024-05-09T08:57:56.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0628000000000003e-05 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748850+0000", 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012666700000000001 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748991+0000", 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014025800000000001 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751743+0000", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0027527300000000001 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752061+0000", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000317616 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752109+0000", 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8313999999999999e-05 2024-05-09T08:57:56.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752154+0000", 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4257e-05 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.301595321999997, 2024-05-09T08:57:56.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0061489200000000004, 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:56.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745521+0000", 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745523+0000", 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0729999999999999e-06 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749310+0000", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003786359 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749318+0000", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3510000000000002e-06 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749893+0000", 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000575207 2024-05-09T08:57:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750020+0000", 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012688399999999999 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750152+0000", 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013194500000000001 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751223+0000", 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010710470000000001 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751571+0000", 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034816699999999998 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751626+0000", 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4688999999999998e-05 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751670+0000", 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3931999999999999e-05 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285619103000002, 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073256839999999998, 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761497+0000", 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761499+0000", 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.821e-06 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761508+0000", 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3999999999999992e-06 2024-05-09T08:57:56.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761574+0000", 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6769999999999999e-05 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761587+0000", 2024-05-09T08:57:56.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2897e-05 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761713+0000", 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012593599999999999 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761887+0000", 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000174315 2024-05-09T08:57:56.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762051+0000", 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000163244 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766108+0000", 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040574959999999998 2024-05-09T08:57:56.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768719+0000", 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002611089 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768775+0000", 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5798999999999999e-05 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768823+0000", 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7917000000000002e-05 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285606400999995, 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007533718, 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.123 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761510+0000", 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2399999999999999e-07 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4819999999999999e-06 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761626+0000", 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011198500000000001 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761633+0000", 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.126e-06 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762428+0000", 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00079549799999999995 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762543+0000", 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114246 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762689+0000", 2024-05-09T08:57:56.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014671500000000001 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767829+0000", 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051401169999999996 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768970+0000", 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001140961 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769005+0000", 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4480000000000002e-05 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.769044+0000", 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8884000000000002e-05 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285598484000005, 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046578410000000002, 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:56.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6810000000000001e-06 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761660+0000", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000140214 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761665+0000", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4280000000000004e-06 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761906+0000", 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000240628 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762056+0000", 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000149467 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762172+0000", 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116907 2024-05-09T08:57:56.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764313+0000", 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021401789999999999 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766066+0000", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00175293 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766127+0000", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1328999999999995e-05 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766176+0000", 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8877e-05 2024-05-09T08:57:56.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285595975999996, 2024-05-09T08:57:56.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067161219999999997, 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761520+0000", 2024-05-09T08:57:56.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761521+0000", 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761522+0000", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.579e-06 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761687+0000", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000165055 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761696+0000", 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9139999999999997e-06 2024-05-09T08:57:56.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762580+0000", 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00088392899999999999 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762675+0000", 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.4728000000000005e-05 2024-05-09T08:57:56.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 1", 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764027+0000", 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013523090000000001 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765016+0000", 2024-05-09T08:57:56.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00098878700000000009 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768144+0000", 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031283330000000001 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768193+0000", 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9088000000000002e-05 2024-05-09T08:57:56.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768237+0000", 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3198999999999998e-05 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:25 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285593512000005, 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047042200000000003, 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761523+0000", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0100000000000001e-07 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761536+0000", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3275000000000001e-05 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761884+0000", 2024-05-09T08:57:56.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00034739299999999999 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761892+0000", 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4700000000000002e-06 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761954+0000", 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1530000000000002e-05 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762089+0000", 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135337 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 2", 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762209+0000", 2024-05-09T08:57:56.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119646 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764580+0000", 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023711000000000001 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766138+0000", 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001557761 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766188+0000", 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0158999999999999e-05 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766227+0000", 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9347999999999997e-05 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285578216999994, 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006594974, 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761538+0000", 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761539+0000", 2024-05-09T08:57:56.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6400000000000003e-07 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761542+0000", 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.835e-06 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:57:56.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038200700000000002 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761929+0000", 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6860000000000003e-06 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762484+0000", 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055466700000000001 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763076+0000", 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00059189300000000002 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763207+0000", 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000130891 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019748130000000002 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768034+0000", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002852434 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768095+0000", 2024-05-09T08:57:56.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0822000000000001e-05 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768133+0000", 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8461999999999999e-05 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:56.158 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285573896000002, 2024-05-09T08:57:56.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067733380000000003, 2024-05-09T08:57:56.159 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.159 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.163 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:56.163 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761543+0000", 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761544+0000", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.3549999999999999e-06 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761945+0000", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040048299999999998 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761950+0000", 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5269999999999998e-06 2024-05-09T08:57:56.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764426+0000", 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0024762619999999999 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764534+0000", 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010727599999999999 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768316+0000", 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.003782171 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.285570755999998, 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063429410000000004, 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.167 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761546+0000", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761548+0000", 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9479999999999998e-06 2024-05-09T08:57:56.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761963+0000", 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00041486799999999999 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761968+0000", 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0529999999999997e-06 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762022+0000", 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3949000000000001e-05 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762158+0000", 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000135931 2024-05-09T08:57:56.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767889+0000", 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057309489999999999 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.4.15:3 2.1c e20/17 2:3d0fce1b:::628.00000000:head v 20'3, mlcod=20'1)", 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.283110956000002, 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039596459999999998, 2024-05-09T08:57:56.172 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764005+0000", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764006+0000", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3900000000000003e-07 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764014+0000", 2024-05-09T08:57:56.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7030000000000006e-06 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764025+0000", 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1044e-05 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764032+0000", 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2389999999999996e-06 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764101+0000", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8145000000000006e-05 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764119+0000", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8546000000000002e-05 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767913+0000", 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037942969999999999 2024-05-09T08:57:56.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767965+0000", 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1733000000000001e-05 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.177 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:56.179 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:56.179 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:56.179+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:56.279 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:56.281+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:56.281 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:56.281 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:56.281 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:56.281 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:56.290 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:56.381 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:56.382+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:56.433 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:56.433 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:56.433 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:56.433 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:56.434 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:56.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:56.483 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:56.484+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:56.582 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:56.584+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:56.586 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.90560141500001, 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:56.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:56.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:56.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.905598548, 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.601 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.90559490299999, 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:56.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:56.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.606 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.862785738, 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:56.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:56.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:56.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 104.857993749, 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:56.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.855487381000003, 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.619 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.854015990999997, 2024-05-09T08:57:56.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:56.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:56.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:56.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 98.853940152999996, 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:56.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:56.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:56.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841118742999996, 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:56.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841097744999999, 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:56.642 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841093350999998, 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:56.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:56.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841088292999999, 2024-05-09T08:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:56.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841084812000005, 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:56.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:56.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841080194, 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:56.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:56.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:56.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.674 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841076724999994, 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:56.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:56.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.841073424000001, 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:56.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:56.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.687 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 92.840760355, 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:56.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:56.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:56.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:56.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.824684532999996, 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:56.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.824679853999996, 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:56.700 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:56.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:56.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.824673836000002, 2024-05-09T08:57:56.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:56.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:56.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:56.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:56.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:56.712 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:56.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:56.714 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:56.714+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:56.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-09T08:57:56.814 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:56.815+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:56.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.857+0000 7f0ccd577640 1 Processor -- start 2024-05-09T08:57:56.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.857+0000 7f0ccd577640 1 -- start start 2024-05-09T08:57:56.859 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.857+0000 7f0ccd577640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cc8155080 0x7f0cc8070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0ccd577640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 0x7f0cc806b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0ccd577640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc80c9fa0 con 0x7f0cc806b740 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0ccd577640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80707f0 con 0x7f0cc8155080 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0ccd577640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80ca280 con 0x7f0cc80c96f0 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cc8155080 0x7f0cc8070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0cc8155080 0x7f0cc8070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53798/0 (socket says 172.21.15.49:53798) 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc77fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0cc806b740 0x7f0cc815e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49460/0 (socket says 172.21.15.49:49460) 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 -- 172.21.15.49:0/2792523989 learned_addr learned my addr 172.21.15.49:0/2792523989 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc67fc640 1 --2- 172.21.15.49:0/2792523989 >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 0x7f0cc806b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 -- 172.21.15.49:0/2792523989 >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 msgr2=0x7f0cc806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 --2- 172.21.15.49:0/2792523989 >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 0x7f0cc806b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:56.860 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 -- 172.21.15.49:0/2792523989 >> v1:172.21.15.49:6789/0 conn(0x7f0cc806b740 legacy=0x7f0cc815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc5ffb640 1 -- 172.21.15.49:0/2792523989 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3362772635 0 0) 0x7f0cc80c9fa0 con 0x7f0cc806b740 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc6ffd640 1 -- 172.21.15.49:0/2792523989 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80ca560 con 0x7f0cc8155080 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.858+0000 7f0cc67fc640 1 --2- 172.21.15.49:0/2792523989 >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 0x7f0cc806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.859+0000 7f0cc6ffd640 1 --2- 172.21.15.49:0/2792523989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc8070140 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8070970 tx=0x7f0cb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c33e323b43c7995 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.859+0000 7f0cc5ffb640 1 -- 172.21.15.49:0/2792523989 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8037070 con 0x7f0cc8155080 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.859+0000 7f0cc5ffb640 1 -- 172.21.15.49:0/2792523989 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb8032440 con 0x7f0cc8155080 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.859+0000 7f0cc5ffb640 1 -- 172.21.15.49:0/2792523989 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8032810 con 0x7f0cc8155080 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 -- 172.21.15.49:0/2792523989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 msgr2=0x7f0cc8070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 --2- 172.21.15.49:0/2792523989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc8070140 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8070970 tx=0x7f0cb802a270 comp rx=0 tx=0).stop 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 -- 172.21.15.49:0/2792523989 shutdown_connections 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 --2- 172.21.15.49:0/2792523989 >> v2:172.21.15.183:3300/0 conn(0x7f0cc80c96f0 0x7f0cc806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 --2- 172.21.15.49:0/2792523989 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc8070140 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:56.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 -- 172.21.15.49:0/2792523989 >> 172.21.15.49:0/2792523989 conn(0x7f0cc8072230 msgr2=0x7f0cc8072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 -- 172.21.15.49:0/2792523989 shutdown_connections 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.860+0000 7f0ccd577640 1 -- 172.21.15.49:0/2792523989 wait complete. 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:56.862 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.861+0000 7f0ccd577640 1 Processor -- start 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.861+0000 7f0ccd577640 1 -- start start 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc806b740 0x7f0cc81f1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80c96f0 0x7f0cc815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8155460 con 0x7f0cc8155080 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80cab20 con 0x7f0cc806b740 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0ccd577640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc806d130 con 0x7f0cc80c96f0 2024-05-09T08:57:56.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53804/0 (socket says 172.21.15.49:53804) 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0cc77fe640 1 -- 172.21.15.49:0/4106144408 learned_addr learned my addr 172.21.15.49:0/4106144408 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0cc67fc640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80c96f0 0x7f0cc815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.862+0000 7f0cc6ffd640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc806b740 0x7f0cc81f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 -- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80c96f0 msgr2=0x7f0cc815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80c96f0 0x7f0cc815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 -- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc806b740 msgr2=0x7f0cc81f1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc806b740 0x7f0cc81f1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 -- 172.21.15.49:0/4106144408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc8148ec0 con 0x7f0cc8155080 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0cc77fe640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c730 tx=0x7f0cc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0018070 con 0x7f0cc8155080 2024-05-09T08:57:56.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc000ce40 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81f1780 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000a880 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.863+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc8152170 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.864+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc804f660 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.865+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cc001b070 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.865+0000 7f0caf7fe640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 0x7f0cb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.866+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5886+0+0 (secure 0 0 0) 0x7f0cc008a440 con 0x7f0cc8155080 2024-05-09T08:57:56.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.866+0000 7f0cc6ffd640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 0x7f0cb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:56.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.866+0000 7f0cc6ffd640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 0x7f0cb406e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:56.869 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:56.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.870+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0cc00586e0 con 0x7f0cc8155080 2024-05-09T08:57:56.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:57:56.953 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:56.954+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:56.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.996+0000 7feb7ed9e640 1 Processor -- start 2024-05-09T08:57:56.995 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.996+0000 7feb7ed9e640 1 -- start start 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7ed9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb780cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7ed9e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feb780c6d50 0x7feb780cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7ed9e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb78072600 con 0x7feb78154f70 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7ed9e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7feb780728e0 con 0x7feb780c6d50 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7ed9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7feb780c5880 con 0x7feb78154b90 2024-05-09T08:57:56.999 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb6ffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7feb78154f70 0x7feb780c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49470/0 (socket says 172.21.15.49:49470) 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb6ffff640 1 -- 172.21.15.49:0/3375501611 learned_addr learned my addr 172.21.15.49:0/3375501611 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7d33b640 1 --2- 172.21.15.49:0/3375501611 >> v2:172.21.15.183:3300/0 conn(0x7feb780c6d50 0x7feb780cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb780cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010156151 0 0) 0x7feb78072600 con 0x7feb78154f70 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb60003520 con 0x7feb78154f70 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.997+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 14891133 0 0) 0x7feb60003520 con 0x7feb78154f70 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 msgr2=0x7feb780cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb780cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 >> v2:172.21.15.183:3300/0 conn(0x7feb780c6d50 msgr2=0x7feb780cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 --2- 172.21.15.49:0/3375501611 >> v2:172.21.15.183:3300/0 conn(0x7feb780c6d50 0x7feb780cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb780c5b60 con 0x7feb78154f70 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7feb68001670 con 0x7feb78154f70 2024-05-09T08:57:57.000 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7d33b640 1 --2- 172.21.15.49:0/3375501611 >> v2:172.21.15.183:3300/0 conn(0x7feb780c6d50 0x7feb780cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb780cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 >> v1:172.21.15.49:6789/0 conn(0x7feb78154f70 legacy=0x7feb780c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb600039b0 0x7feb60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb60004d40 0x7feb600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feb780728e0 con 0x7feb600039b0 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb780c5880 con 0x7feb60004d40 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7feb6000b9c0 con 0x7feb78154b90 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb600039b0 0x7feb60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7d33b640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb60004d40 0x7feb600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb6ffff640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 -- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 msgr2=0x7feb60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:57.001 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb78154b90 0x7feb60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 -- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb60004d40 msgr2=0x7feb600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb60004d40 0x7feb600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7cb3a640 1 -- 172.21.15.49:0/3375501611 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb60003520 con 0x7feb600039b0 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.998+0000 7feb7d33b640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb60004d40 0x7feb600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb600039b0 0x7feb60003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e350 tx=0x7feb60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2664fb79c5e0884d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74033070 con 0x7feb600039b0 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7402f660 con 0x7feb600039b0 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb6f7fe640 1 -- 172.21.15.49:0/3375501611 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74009de0 con 0x7feb600039b0 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb7ed9e640 1 -- 172.21.15.49:0/3375501611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb600039b0 msgr2=0x7feb60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/3375501611 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb600039b0 0x7feb60003dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feb7400e350 tx=0x7feb60005830 comp rx=0 tx=0).stop 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:56.999+0000 7feb7d33b640 1 -- 172.21.15.49:0/3375501611 reap_dead start 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.000+0000 7feb7ed9e640 1 -- 172.21.15.49:0/3375501611 shutdown_connections 2024-05-09T08:57:57.002 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.000+0000 7feb7ed9e640 1 -- 172.21.15.49:0/3375501611 >> 172.21.15.49:0/3375501611 conn(0x7feb78074840 msgr2=0x7feb78074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:57.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.000+0000 7feb7ed9e640 1 -- 172.21.15.49:0/3375501611 shutdown_connections 2024-05-09T08:57:57.003 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.000+0000 7feb7ed9e640 1 -- 172.21.15.49:0/3375501611 wait complete. 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.001+0000 7feb7ed9e640 1 Processor -- start 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.001+0000 7feb7ed9e640 1 -- start start 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb600039b0 0x7feb780cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb78151f30 0x7feb781f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb7814d5a0 0x7feb781f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.006 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7feb7815af40 con 0x7feb78151f30 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb780c6120 con 0x7feb7814d5a0 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7ed9e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7feb780cb280 con 0x7feb600039b0 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb78151f30 0x7feb781f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb78151f30 0x7feb781f3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53820/0 (socket says 172.21.15.49:53820) 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 -- 172.21.15.49:0/2180616920 learned_addr learned my addr 172.21.15.49:0/2180616920 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb600039b0 0x7feb780cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb7814d5a0 0x7feb781f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 -- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb600039b0 msgr2=0x7feb780cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb600039b0 0x7feb780cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 -- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb7814d5a0 msgr2=0x7feb781f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb7814d5a0 0x7feb781f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.007 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7814db90 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb7814d5a0 0x7feb781f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.002+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7feb78151f30 0x7feb781f3de0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb68007370 tx=0x7feb68007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6801a070 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6800f9c0 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800fd90 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb78153380 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb78153940 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.003+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7feb7804f660 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.004+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb6801d070 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb6d7fa640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7feb6406be60 0x7feb6406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5886+0+0 (secure 0 0 0) 0x7feb6808a710 con 0x7feb78151f30 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7feb6406be60 0x7feb6406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.008 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7feb68090170 con 0x7feb78151f30 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7feb78149e50 0x7feb7814a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb780cbde0 con 0x7feb78149e50 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7feb78149e50 0x7feb7814a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7feb6406be60 0x7feb6406e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb74035000 tx=0x7feb7402f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.005+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7feb78149e50 0x7feb7814a230 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.009 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.006+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb780cbde0 con 0x7feb78149e50 2024-05-09T08:57:57.013 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:57.013 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:57.013 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:57.013 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:57.013 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:57.016 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:57.017+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:57.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.021+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb780ce390 con 0x7feb78149e50 2024-05-09T08:57:57.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:57:57.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.023+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb780ce390 con 0x7feb78149e50 2024-05-09T08:57:57.023 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.023+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7feb780b5bc0 0x7feb780b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.023+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7feb780cb780 con 0x7feb780b5bc0 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.023+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7feb780b5bc0 0x7feb780b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.024+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7feb780b5bc0 0x7feb780b5fa0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.025+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb780cb780 con 0x7feb780b5bc0 2024-05-09T08:57:57.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.040+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb7814acf0 con 0x7feb780b5bc0 2024-05-09T08:57:57.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb7814acf0 con 0x7feb780b5bc0 2024-05-09T08:57:57.041 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:57.042 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=57}) -- 0x7feb7814a770 con 0x7feb78151f30 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb781557d0 con 0x7feb78151f30 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7feb68091070 con 0x7feb78151f30 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7feb78062540 0x7feb780cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.042+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7feb781559b0 con 0x7feb78062540 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.043+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7feb78062540 0x7feb780cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.043+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7feb78062540 0x7feb780cc5e0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.043+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb781559b0 con 0x7feb78062540 2024-05-09T08:57:57.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.058+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb78062bc0 con 0x7feb78062540 2024-05-09T08:57:57.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.060+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb78062bc0 con 0x7feb78062540 2024-05-09T08:57:57.059 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:57.059 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:57.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.060+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7feb7802c040 0x7feb7802c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.060+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7feb780cd3f0 con 0x7feb7802c040 2024-05-09T08:57:57.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.060+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7feb7802c040 0x7feb7802c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.060+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7feb7802c040 0x7feb7802c4a0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.061+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb780cd3f0 con 0x7feb7802c040 2024-05-09T08:57:57.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.067+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f0cc8070010 con 0x7f0cc8155080 2024-05-09T08:57:57.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.076+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb7802c9e0 con 0x7feb7802c040 2024-05-09T08:57:57.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.077+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb7802c9e0 con 0x7feb7802c040 2024-05-09T08:57:57.076 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:57.077 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:57.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.077+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7feb7802d060 0x7feb78205160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.077+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7feb7802d4c0 con 0x7feb7802d060 2024-05-09T08:57:57.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.078+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7feb7802d060 0x7feb78205160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.078+0000 7feb7cb3a640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7feb7802d060 0x7feb78205160 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.078 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.079+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb7802d4c0 con 0x7feb7802d060 2024-05-09T08:57:57.092 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.093+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb7802d4c0 con 0x7feb7802d060 2024-05-09T08:57:57.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.096+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb7802d4c0 con 0x7feb7802d060 2024-05-09T08:57:57.095 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:57.096 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:57.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.096+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7feb78205bb0 0x7feb78207fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.096+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7feb780cd560 con 0x7feb78205bb0 2024-05-09T08:57:57.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.096+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7feb78205bb0 0x7feb78207fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.096+0000 7feb6ffff640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7feb78205bb0 0x7feb78207fa0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.097 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.097+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb780cd560 con 0x7feb78205bb0 2024-05-09T08:57:57.110 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.111+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb780cd560 con 0x7feb78205bb0 2024-05-09T08:57:57.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.113+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb780cd560 con 0x7feb78205bb0 2024-05-09T08:57:57.112 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:57.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:57.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.113+0000 7feb7ed9e640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7feb782089f0 0x7feb7820ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:57.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.113+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7feb7820dbb0 con 0x7feb782089f0 2024-05-09T08:57:57.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.114+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7feb782089f0 0x7feb7820ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:57.113 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.114+0000 7feb7d33b640 1 --2- 172.21.15.49:0/2180616920 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7feb782089f0 0x7feb7820ade0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:57.114 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.115+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb7820dbb0 con 0x7feb782089f0 2024-05-09T08:57:57.118 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:57.119+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:57.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.129+0000 7feb7ed9e640 1 -- 172.21.15.49:0/2180616920 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7feb7820b3a0 con 0x7feb782089f0 2024-05-09T08:57:57.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.130+0000 7feb6d7fa640 1 -- 172.21.15.49:0/2180616920 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7feb7820b3a0 con 0x7feb782089f0 2024-05-09T08:57:57.130 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:57.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:57.134 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:57.134 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:57.167 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.167 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.176 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:57:57.219 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:57.220+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:57.287 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.288 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:57:57.341 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:57.342+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:57.407 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.408 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.418 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:57:57.459 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:57.460+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:57.522 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:57.524+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:57.569 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.570 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.579 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:57:57.623 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:57.625+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:57.689 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.689 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.699 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:57:57.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.727+0000 7f0caf7fe640 1 -- 172.21.15.49:0/4106144408 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v57) ==== 97+0+0 (secure 0 0 0) 0x7f0cc005c9e0 con 0x7f0cc8155080 2024-05-09T08:57:57.726 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-05-09T08:57:57.726 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:57.727+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.730+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 msgr2=0x7f0cb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.730+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 0x7f0cb406e320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8034000 tx=0x7f0cb802e000 comp rx=0 tx=0).stop 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.730+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 msgr2=0x7f0cc80caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.730+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c730 tx=0x7f0cc000cbf0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 shutdown_connections 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0cb406be60 0x7f0cb406e320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.730 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0cc8155080 0x7f0cc80caf60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0cc80c96f0 0x7f0cc815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 --2- 172.21.15.49:0/4106144408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0cc806b740 0x7f0cc81f1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:57.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 >> 172.21.15.49:0/4106144408 conn(0x7f0cc8072230 msgr2=0x7f0cc8159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:57.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 shutdown_connections 2024-05-09T08:57:57.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:57.731+0000 7f0ccd577640 1 -- 172.21.15.49:0/4106144408 wait complete. 2024-05-09T08:57:57.818 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.818 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.827 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:57.828+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:57.828 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:57:57.928 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:57.929+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:57.979 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:57.979 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:57.989 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:57:58.029 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:58.031+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:58.093 INFO:tasks.workunit.client.0.smithi049.stderr:Updating files: 96% (12075/12535) Updating files: 97% (12159/12535) Updating files: 98% (12285/12535) Updating files: 99% (12410/12535) Updating files: 100% (12535/12535) Updating files: 100% (12535/12535), done. 2024-05-09T08:57:58.130 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:58.131+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:58.144 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.145 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:57:58.232 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:58.233+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:58.305 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.305 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T08:57:58.358 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: git switch -c 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr:Or undo this operation with: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: git switch - 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr: 2024-05-09T08:57:58.359 INFO:tasks.workunit.client.0.smithi049.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T08:57:58.359 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:58.359+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:58.362 DEBUG:teuthology.orchestra.run.smithi049:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T08:57:58.425 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.426 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.428 INFO:tasks.workunit.client.0.smithi049.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T08:57:58.430 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:57:58.430 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T08:57:58.434 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:58.435+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:58.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:57:58.535 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:58.536+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:58.549 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.549 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.558 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:57:58.635 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:58.637+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:58.703 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.703 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:57:58.756 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:57:58.757+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:57:58.818 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.818 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:57:58.837 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:58.839+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:58.939 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:57:58.940+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:57:58.969 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:58.969 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:58.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:57:59.034 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T08:57:59.039 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:57:59.040+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:57:59.080 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:59.080 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:59.088 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:57:59.097 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T08:57:59.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-09T08:57:59.139 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T08:57:59.140 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:57:59.141 INFO:tasks.workunit.client.0.smithi049.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T08:57:59.173 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:57:59.174+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:57:59.182 INFO:tasks.workunit.client.0.smithi049.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T08:57:59.186 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:57:59.186 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T08:57:59.231 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:57:59.231 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:57:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.240+0000 7fd6824d4640 1 Processor -- start 2024-05-09T08:57:59.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.240+0000 7fd6824d4640 1 -- start start 2024-05-09T08:57:59.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd6824d4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd67c154b90 0x7fd67c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd6824d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c0c6d50 0x7fd67c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd6824d4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd67c072600 con 0x7fd67c154f70 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd6824d4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd67c0728e0 con 0x7fd67c154b90 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd6824d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c0c5880 con 0x7fd67c0c6d50 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd67c154b90 0x7fd67c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67b7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd67c154f70 0x7fd67c0c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49484/0 (socket says 172.21.15.49:49484) 2024-05-09T08:57:59.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd67c154b90 0x7fd67c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53832/0 (socket says 172.21.15.49:53832) 2024-05-09T08:57:59.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67b7fe640 1 -- 172.21.15.49:0/4248767855 learned_addr learned my addr 172.21.15.49:0/4248767855 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:57:59.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd680a71640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c0c6d50 0x7fd67c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.243 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:59.241+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 -- 172.21.15.49:0/4248767855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c0c6d50 msgr2=0x7fd67c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c0c6d50 0x7fd67c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 -- 172.21.15.49:0/4248767855 >> v1:172.21.15.49:6789/0 conn(0x7fd67c154f70 legacy=0x7fd67c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67bfff640 1 -- 172.21.15.49:0/4248767855 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c0c5b60 con 0x7fd67c154b90 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.241+0000 7fd67affd640 1 -- 172.21.15.49:0/4248767855 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228716641 0 0) 0x7fd67c072600 con 0x7fd67c154f70 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.242+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c154b90 0x7fd67c0cb280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0097a0 tx=0x7fd66c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04bcc0f7f585121 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.242+0000 7fd67affd640 1 -- 172.21.15.49:0/4248767855 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c037070 con 0x7fd67c154b90 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.242+0000 7fd67affd640 1 -- 172.21.15.49:0/4248767855 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c032440 con 0x7fd67c154b90 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.242+0000 7fd67affd640 1 -- 172.21.15.49:0/4248767855 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c032810 con 0x7fd67c154b90 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 -- 172.21.15.49:0/4248767855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c154b90 msgr2=0x7fd67c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c154b90 0x7fd67c0cb280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0097a0 tx=0x7fd66c02a210 comp rx=0 tx=0).stop 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 -- 172.21.15.49:0/4248767855 shutdown_connections 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c0c6d50 0x7fd67c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4248767855 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c154b90 0x7fd67c0cb280 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 -- 172.21.15.49:0/4248767855 >> 172.21.15.49:0/4248767855 conn(0x7fd67c074840 msgr2=0x7fd67c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 -- 172.21.15.49:0/4248767855 shutdown_connections 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.243+0000 7fd6824d4640 1 -- 172.21.15.49:0/4248767855 wait complete. 2024-05-09T08:57:59.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.244+0000 7fd6824d4640 1 Processor -- start 2024-05-09T08:57:59.248 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.244+0000 7fd6824d4640 1 -- start start 2024-05-09T08:57:59.248 INFO:tasks.workunit:Running workunits matching fs/misc/trivial_sync.sh on client.0... 2024-05-09T08:57:59.249 INFO:tasks.workunit:Running workunit fs/misc/trivial_sync.sh... 2024-05-09T08:57:59.250 DEBUG:teuthology.orchestra.run.smithi049:workunit test fs/misc/trivial_sync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c0c6d50 0x7fd67c0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c154b90 0x7fd67c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd67c154f70 0x7fd67c0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c15af40 con 0x7fd67c0c6d50 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c0c6120 con 0x7fd67c154f70 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd6824d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd67c071820 con 0x7fd67c154b90 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c0c6d50 0x7fd67c0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c0c6d50 0x7fd67c0cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53850/0 (socket says 172.21.15.49:53850) 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 -- 172.21.15.49:0/4122073524 learned_addr learned my addr 172.21.15.49:0/4122073524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c154b90 0x7fd67c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd67c154f70 0x7fd67c0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 -- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c154b90 msgr2=0x7fd67c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd67c154b90 0x7fd67c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 -- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd67c154f70 msgr2=0x7fd67c0c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd67c154f70 0x7fd67c0c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd67bfff640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c154180 con 0x7fd67c0c6d50 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.245+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd67c154f70 0x7fd67c0c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd67c0c6d50 0x7fd67c0cf650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0097a0 tx=0x7fd66c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c037050 con 0x7fd67c0c6d50 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd66c038040 con 0x7fd67c0c6d50 2024-05-09T08:57:59.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c1f0950 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004490 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.246+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1f0ef0 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.247+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd67c153290 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.247+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd66c03c6d0 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd678ff9640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd66806be60 0x7fd66806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5886+0+0 (secure 0 0 0) 0x7fd66c0ac260 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd66806be60 0x7fd66806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v57) ==== 54+0+16 (secure 0 0 0) 0x7fd67c153290 con 0x7fd67c0c6d50 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd67c0c81d0 0x7fd67c1536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd67c04f660 con 0x7fd67c0c81d0 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.248+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd67c0c81d0 0x7fd67c1536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.249+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd66806be60 0x7fd66806e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0c8ac0 tx=0x7fd670007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.249+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fd67c0c81d0 0x7fd67c1536b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.249+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c04f660 con 0x7fd67c0c81d0 2024-05-09T08:57:59.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.265+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c074fa0 con 0x7fd67c0c81d0 2024-05-09T08:57:59.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.266+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c074fa0 con 0x7fd67c0c81d0 2024-05-09T08:57:59.266 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:57:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:57:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.267+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd67c14a380 0x7fd67c1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.267+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd67c14a7c0 con 0x7fd67c14a380 2024-05-09T08:57:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.267+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd67c14a380 0x7fd67c1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.267+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fd67c14a380 0x7fd67c1f3d40 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.268+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c14a7c0 con 0x7fd67c14a380 2024-05-09T08:57:59.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.285+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c14abd0 con 0x7fd67c14a380 2024-05-09T08:57:59.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.287+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c14abd0 con 0x7fd67c14a380 2024-05-09T08:57:59.286 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:57:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:57:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.287+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=58}) -- 0x7fd67c151c40 con 0x7fd67c0c6d50 2024-05-09T08:57:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.287+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd67c1f4280 con 0x7fd67c0c6d50 2024-05-09T08:57:59.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.287+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd66c079e50 con 0x7fd67c0c6d50 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.288+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd67c061c70 0x7fd67c0620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.288+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fd67c0ccf40 con 0x7fd67c061c70 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.288+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd67c061c70 0x7fd67c0620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.288+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7fd67c061c70 0x7fd67c0620b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.289+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c0ccf40 con 0x7fd67c061c70 2024-05-09T08:57:59.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.303+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c0628c0 con 0x7fd67c061c70 2024-05-09T08:57:59.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.305+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c0628c0 con 0x7fd67c061c70 2024-05-09T08:57:59.304 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:57:59.305 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:57:59.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.305+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd67c0cc2d0 0x7fd67c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.305+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7fd67c062840 con 0x7fd67c0cc2d0 2024-05-09T08:57:59.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.305+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd67c0cc2d0 0x7fd67c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.306+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7fd67c0cc2d0 0x7fd67c0cc6f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.306+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c062840 con 0x7fd67c0cc2d0 2024-05-09T08:57:59.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.321+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c062840 con 0x7fd67c0cc2d0 2024-05-09T08:57:59.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.322+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c062840 con 0x7fd67c0cc2d0 2024-05-09T08:57:59.322 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:57:59.322 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:57:59.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.323+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd67c02c550 0x7fd67c02c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.323+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7fd67c0ccf60 con 0x7fd67c02c550 2024-05-09T08:57:59.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.323+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd67c02c550 0x7fd67c02c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.323+0000 7fd67bfff640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7fd67c02c550 0x7fd67c02c950 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.324+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c1f4280 con 0x7fd67c02c550 2024-05-09T08:57:59.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.338+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c1f4280 con 0x7fd67c02c550 2024-05-09T08:57:59.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.340+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c1f4280 con 0x7fd67c02c550 2024-05-09T08:57:59.339 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:57:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:57:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.340+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd67c2050c0 0x7fd67c2074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.340+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fd67c02d3a0 con 0x7fd67c2050c0 2024-05-09T08:57:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.340+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd67c2050c0 0x7fd67c2074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.341+0000 7fd67b7fe640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7fd67c2050c0 0x7fd67c2074b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.341+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c02d3a0 con 0x7fd67c2050c0 2024-05-09T08:57:59.341 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:59.342+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:59.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.356+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c02d3a0 con 0x7fd67c2050c0 2024-05-09T08:57:59.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.357+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c02d3a0 con 0x7fd67c2050c0 2024-05-09T08:57:59.357 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:57:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:57:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.357+0000 7fd6824d4640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd67c207f00 0x7fd67c20a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.358+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fd67c20d140 con 0x7fd67c207f00 2024-05-09T08:57:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.358+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd67c207f00 0x7fd67c20a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.358+0000 7fd680a71640 1 --2- 172.21.15.49:0/4122073524 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fd67c207f00 0x7fd67c20a2f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.359+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd67c20d140 con 0x7fd67c207f00 2024-05-09T08:57:59.372 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.373+0000 7fd6824d4640 1 -- 172.21.15.49:0/4122073524 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fd67c20ab00 con 0x7fd67c207f00 2024-05-09T08:57:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.375+0000 7fd678ff9640 1 -- 172.21.15.49:0/4122073524 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd67c20ab00 con 0x7fd67c207f00 2024-05-09T08:57:59.374 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:57:59.374 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:57:59.378 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:57:59.378 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:57:59.384 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:59.384 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:59.384 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:59.384 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:59.393 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:57:59.443 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:57:59.444+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:57:59.495 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:59.496 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:59.496 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:59.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:57:59.544 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:59.545+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:59.608 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:57:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.92770358200001, 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051248379999999996, 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680804+0000", 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.99e-07 2024-05-09T08:57:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680806+0000", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.863e-06 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681077+0000", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00027095899999999998 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681085+0000", 2024-05-09T08:57:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3140000000000004e-06 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682063+0000", 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0009777430000000001 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682189+0000", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012635200000000001 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682307+0000", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118009 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684350+0000", 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020429910000000001 2024-05-09T08:57:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685854+0000", 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001503643 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685903+0000", 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9076000000000002e-05 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685929+0000", 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5689000000000001e-05 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.927700715, 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052441969999999999, 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680807+0000", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.04e-07 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680809+0000", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6670000000000001e-06 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681104+0000", 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00029569299999999998 2024-05-09T08:57:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681124+0000", 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9908999999999999e-05 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682730+0000", 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016061370000000001 2024-05-09T08:57:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.682854+0000", 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012352700000000001 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683015+0000", 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000161018 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.684950+0000", 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001935038 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.685981+0000", 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010314479999999999 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686017+0000", 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5995999999999999e-05 2024-05-09T08:57:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686051+0000", 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3559999999999997e-05 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:57:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.92769706999999, 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053301900000000003, 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680810+0000", 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680811+0000", 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1899999999999999e-07 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.680812+0000", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0809999999999999e-06 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681140+0000", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00032827699999999997 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.681145+0000", 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0239999999999997e-06 2024-05-09T08:57:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683437+0000", 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022922820000000001 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.683566+0000", 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012831100000000001 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:06.686141+0000", 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002574996 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:9 2.14 2:2e2b5214:::610.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.884887905, 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011128466, 2024-05-09T08:57:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723619+0000", 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723620+0000", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6699999999999999e-07 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723623+0000", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7300000000000001e-06 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723816+0000", 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000193196 2024-05-09T08:57:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723823+0000", 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9349999999999997e-06 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723869+0000", 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6495999999999997e-05 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.723970+0000", 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000100626 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724052+0000", 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.2047999999999998e-05 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.724592+0000", 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00053989100000000003 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734693+0000", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010100866 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734725+0000", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2679999999999999e-05 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.734748+0000", 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2631000000000001e-05 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.1.9:29 2.14 2:2e2b5214:::610.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.880095916, 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073710499999999997, 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728411+0000", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728412+0000", 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4299999999999999e-07 2024-05-09T08:57:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728420+0000", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8159999999999994e-06 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728456+0000", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6732e-05 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728465+0000", 2024-05-09T08:57:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7339999999999997e-06 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728513+0000", 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8031999999999997e-05 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728596+0000", 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3090999999999994e-05 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.728706+0000", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010953700000000001 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.729474+0000", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076883599999999996 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735695+0000", 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0062203620000000001 2024-05-09T08:57:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735747+0000", 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2386999999999998e-05 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:11.735783+0000", 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5280000000000001e-05 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.2.11:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 101.877589548, 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069025120000000004, 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730918+0000", 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9999999999999999e-07 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.730920+0000", 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9130000000000001e-06 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731474+0000", 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000554109 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.731483+0000", 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7569999999999993e-06 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732255+0000", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077215599999999997 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013451700000000001 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737820+0000", 2024-05-09T08:57:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054307599999999998 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:9 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 101.876118158, 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0054249839999999999, 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732389+0000", 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732390+0000", 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6600000000000005e-07 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732399+0000", 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8710000000000003e-06 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732409+0000", 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.7690000000000008e-06 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732416+0000", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5839999999999997e-06 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732479+0000", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.279e-05 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732515+0000", 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6202000000000003e-05 2024-05-09T08:57:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737765+0000", 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0052500910000000001 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737814+0000", 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8810999999999997e-05 2024-05-09T08:57:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.2.11:10 2.6 e20/17 2:62978f0a:::617.00000000:head v 20'3, mlcod=0'0)", 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 101.87604232, 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053925930000000002, 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732465+0000", 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732466+0000", 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7000000000000005e-07 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732488+0000", 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2376999999999999e-05 2024-05-09T08:57:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732503+0000", 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4959e-05 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732510+0000", 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1999999999999997e-06 2024-05-09T08:57:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732755+0000", 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00024444899999999999 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.732773+0000", 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8593e-05 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737818+0000", 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0050451339999999997 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:17.737858+0000", 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9310999999999997e-05 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:3 2.1a 2:5fdc322d:::61e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863220909999995, 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059899480000000001, 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745286+0000", 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745290+0000", 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1089999999999998e-06 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745303+0000", 2024-05-09T08:57:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.329e-05 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745358+0000", 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5510999999999998e-05 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745371+0000", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2138e-05 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745460+0000", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9899000000000001e-05 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745632+0000", 2024-05-09T08:57:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000171762 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745789+0000", 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015688 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750053+0000", 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042640500000000001 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751180+0000", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011267110000000001 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751230+0000", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0549e-05 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751276+0000", 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6048999999999999e-05 2024-05-09T08:57:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863199911999999, 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066548880000000003, 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9700000000000003e-07 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.694e-06 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745410+0000", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.9815999999999994e-05 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745416+0000", 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9429999999999997e-06 2024-05-09T08:57:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746132+0000", 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071570900000000003 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746237+0000", 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0001048 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746358+0000", 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121308 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751055+0000", 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0046970859999999996 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751860+0000", 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00080531000000000005 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751918+0000", 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7732999999999998e-05 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751962+0000", 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4192e-05 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863195517999998, 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0067868540000000002, 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3300000000000001e-07 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6720000000000002e-06 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745430+0000", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00011469 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:57:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746609+0000", 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011739840000000001 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746737+0000", 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012814200000000001 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746858+0000", 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120386 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751061+0000", 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0042028910000000003 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752016+0000", 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00095565899999999998 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752059+0000", 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3176999999999997e-05 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752099+0000", 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9249000000000003e-05 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863190459999998, 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069090050000000002, 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745317+0000", 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.48e-07 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745319+0000", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0439999999999998e-06 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745482+0000", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162802 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745491+0000", 2024-05-09T08:57:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2289999999999992e-06 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747604+0000", 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021131990000000001 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747685+0000", 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0458999999999997e-05 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747817+0000", 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013218600000000001 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751065+0000", 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0032480030000000002 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752147+0000", 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010823409999999999 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752188+0000", 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0979999999999997e-05 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752226+0000", 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7514e-05 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:19 2.11 2:890f50f7:::626.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863186979000005, 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0059854230000000001, 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745320+0000", 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745321+0000", 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745324+0000", 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2389999999999998e-06 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745512+0000", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000188483 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745518+0000", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9529999999999996e-06 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745583+0000", 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4895999999999997e-05 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745746+0000", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000162948 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745915+0000", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000169035 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751021+0000", 2024-05-09T08:57:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051060250000000001 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751210+0000", 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018921899999999999 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751260+0000", 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9375000000000001e-05 2024-05-09T08:57:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751306+0000", 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6019000000000001e-05 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863182361, 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066276520000000004, 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745325+0000", 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2999999999999999e-07 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745327+0000", 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745564+0000", 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00023622699999999999 2024-05-09T08:57:59.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745573+0000", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8729999999999996e-06 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746304+0000", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00073185800000000005 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746388+0000", 2024-05-09T08:57:59.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3821999999999998e-05 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 4", 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746527+0000", 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138271 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751027+0000", 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.004500876 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751846+0000", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081817000000000001 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751908+0000", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2487e-05 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751953+0000", 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4619000000000001e-05 2024-05-09T08:57:59.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:1 2.17 2:eb104683:::203.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863178891999993, 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068568479999999996, 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.691 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745328+0000", 2024-05-09T08:57:59.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745329+0000", 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.28e-07 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745331+0000", 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.013e-06 2024-05-09T08:57:59.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745591+0000", 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00026045400000000002 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745598+0000", 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6789999999999999e-06 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746200+0000", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060210199999999998 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746315+0000", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115351 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746437+0000", 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000121616 2024-05-09T08:57:59.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751337+0000", 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048998879999999998 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752091+0000", 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000753724 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752143+0000", 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2914000000000002e-05 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752185+0000", 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1878999999999997e-05 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.697 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.863175591000001, 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060985800000000001, 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745332+0000", 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2700000000000001e-07 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.871e-06 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745620+0000", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000286502 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745625+0000", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9010000000000002e-06 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746808+0000", 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011824889999999999 2024-05-09T08:57:59.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.746933+0000", 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561000000000001 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747056+0000", 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000122423 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750419+0000", 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033626960000000001 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751342+0000", 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00092378600000000001 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751391+0000", 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8820999999999999e-05 2024-05-09T08:57:59.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751430+0000", 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9254000000000001e-05 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_repop(mds.3.13:7 2.6 e20/17 2:6642f7a8:::620.00000000:head v 20'4, mlcod=20'2)", 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 95.862862522, 2024-05-09T08:57:59.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0053773919999999999, 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745645+0000", 2024-05-09T08:57:59.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745646+0000", 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.11e-06 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745657+0000", 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0603e-05 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745665+0000", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7190000000000007e-06 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745671+0000", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.361e-06 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745730+0000", 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8956000000000003e-05 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745758+0000", 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8313000000000001e-05 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.750937+0000", 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051786189999999998 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.751022+0000", 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.5711000000000007e-05 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 89.846786699999996, 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068406430000000004, 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.708 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761721+0000", 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.01e-07 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761724+0000", 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.791e-06 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761827+0000", 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000103176 2024-05-09T08:57:59.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761834+0000", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0999999999999998e-06 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764643+0000", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00280927 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764724+0000", 2024-05-09T08:57:59.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0928999999999997e-05 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 3", 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764867+0000", 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014242599999999999 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767147+0000", 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022803099999999998 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768470+0000", 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001323262 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768517+0000", 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6251000000000002e-05 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768561+0000", 2024-05-09T08:57:59.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4727000000000001e-05 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 89.846782020999996, 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0068451249999999996, 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.714 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761725+0000", 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761726+0000", 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5800000000000001e-07 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761730+0000", 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.51e-06 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763400+0000", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001670337 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763410+0000", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.285e-06 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763494+0000", 2024-05-09T08:57:59.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3959e-05 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763610+0000", 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000116667 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763726+0000", 2024-05-09T08:57:59.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115155 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767569+0000", 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0038432340000000001 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768435+0000", 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00086580499999999998 2024-05-09T08:57:59.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768512+0000", 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7089999999999995e-05 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768570+0000", 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8825e-05 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.720 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 89.846776003000002, 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069965110000000004, 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:57:59.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761731+0000", 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761732+0000", 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8099999999999999e-07 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4640000000000002e-06 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763455+0000", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001719691 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763462+0000", 2024-05-09T08:57:59.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1500000000000002e-06 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764020+0000", 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00055754899999999996 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764092+0000", 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2625000000000003e-05 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 7", 2024-05-09T08:57:59.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764223+0000", 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013090899999999999 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768236+0000", 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040125250000000003 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768615+0000", 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000379627 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768685+0000", 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9672999999999995e-05 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768728+0000", 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3016999999999998e-05 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.726 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:57:59.727 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:57:59.727 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:59.728 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:57:59.729 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:59.729+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:59.747 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:57:59.829 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:57:59.830+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:57:59.833 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:59.833 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:59.833 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-09T08:57:59.833 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:59.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:57:59.852 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.853+0000 7fcf90bb0640 1 Processor -- start 2024-05-09T08:57:59.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.853+0000 7fcf90bb0640 1 -- start start 2024-05-09T08:57:59.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf90bb0640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf8c154f70 0x7fcf8c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf90bb0640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf8c0c6b80 0x7fcf8c071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf90bb0640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8c0c6fd0 con 0x7fcf8c154b90 2024-05-09T08:57:59.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf90bb0640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8c072600 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf90bb0640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8c0728e0 con 0x7fcf8c154f70 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf8ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf8c0c6b80 0x7fcf8c071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf8ad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf8c0c6b80 0x7fcf8c071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53852/0 (socket says 172.21.15.49:53852) 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf8ad9d640 1 -- 172.21.15.49:0/573539062 learned_addr learned my addr 172.21.15.49:0/573539062 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf8a59c640 1 --1- 172.21.15.49:0/573539062 >> v1:172.21.15.183:6789/0 conn(0x7fcf8c154b90 0x7fcf8c0cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:48766/0 (socket says 172.21.15.49:48766) 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.854+0000 7fcf89d9b640 1 --2- 172.21.15.49:0/573539062 >> v2:172.21.15.183:3300/0 conn(0x7fcf8c154f70 0x7fcf8c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8ad9d640 1 -- 172.21.15.49:0/573539062 >> v2:172.21.15.183:3300/0 conn(0x7fcf8c154f70 msgr2=0x7fcf8c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:57:59.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8ad9d640 1 --2- 172.21.15.49:0/573539062 >> v2:172.21.15.183:3300/0 conn(0x7fcf8c154f70 0x7fcf8c0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8ad9d640 1 -- 172.21.15.49:0/573539062 >> v1:172.21.15.183:6789/0 conn(0x7fcf8c154b90 legacy=0x7fcf8c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8ad9d640 1 -- 172.21.15.49:0/573539062 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c0c5880 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8ad9d640 1 --2- 172.21.15.49:0/573539062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c071820 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0cdd20 tx=0x7fcf7c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca039b8d3da77d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8959a640 1 -- 172.21.15.49:0/573539062 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c036070 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8959a640 1 -- 172.21.15.49:0/573539062 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c031600 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.855+0000 7fcf8959a640 1 -- 172.21.15.49:0/573539062 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0319f0 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 -- 172.21.15.49:0/573539062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 msgr2=0x7fcf8c071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/573539062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c071820 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0cdd20 tx=0x7fcf7c02a7c0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 -- 172.21.15.49:0/573539062 shutdown_connections 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/573539062 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c071820 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/573539062 >> v2:172.21.15.183:3300/0 conn(0x7fcf8c154f70 0x7fcf8c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.865 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 -- 172.21.15.49:0/573539062 >> 172.21.15.49:0/573539062 conn(0x7fcf8c074840 msgr2=0x7fcf8c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:57:59.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 -- 172.21.15.49:0/573539062 shutdown_connections 2024-05-09T08:57:59.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.856+0000 7fcf90bb0640 1 -- 172.21.15.49:0/573539062 wait complete. 2024-05-09T08:57:59.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.857+0000 7fcf90bb0640 1 Processor -- start 2024-05-09T08:57:59.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.857+0000 7fcf90bb0640 1 -- start start 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf8c154b90 0x7fcf8c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf8c154f70 0x7fcf8c0b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c0c6120 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c0c5e40 con 0x7fcf8c154f70 2024-05-09T08:57:59.867 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf90bb0640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8c15af40 con 0x7fcf8c154b90 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf8a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf8a59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53858/0 (socket says 172.21.15.49:53858) 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf8a59c640 1 -- 172.21.15.49:0/2100973065 learned_addr learned my addr 172.21.15.49:0/2100973065 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf8ad9d640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf8c154f70 0x7fcf8c0b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.858+0000 7fcf89d9b640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf8c154b90 0x7fcf8c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 -- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf8c154b90 msgr2=0x7fcf8c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf8c154b90 0x7fcf8c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 -- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf8c154f70 msgr2=0x7fcf8c0b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf8c154f70 0x7fcf8c0b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 -- 172.21.15.49:0/2100973065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c1f5320 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf8a59c640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c3f0 tx=0x7fcf8400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf840074e0 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf84007680 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c1540e0 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004a00 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.859+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c152670 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.860+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c04f660 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.861+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf8401a020 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.861+0000 7fcf82ffd640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 0x7fcf6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:57:59.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.861+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5886+0+0 (secure 0 0 0) 0x7fcf8408a4e0 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.861+0000 7fcf89d9b640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 0x7fcf6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:57:59.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.862+0000 7fcf89d9b640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 0x7fcf6c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0d0110 tx=0x7fcf74009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:57:59.871 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:57:59.865+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf84058820 con 0x7fcf8c0c6b80 2024-05-09T08:57:59.929 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:57:59.930+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:57:59.945 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:57:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-09T08:57:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 180, 2024-05-09T08:57:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-09T08:57:59.945 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:57:59.953 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:58:00.031 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-05-09T08:58:00.032+0000 7f2297b5b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 35657) UID: 0 2024-05-09T08:58:00.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.056+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fcf8c158b00 con 0x7fcf8c0c6b80 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.436225939, 2024-05-09T08:58:00.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.16306357099999999, 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.101 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663497+0000", 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663499+0000", 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9410000000000001e-06 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663507+0000", 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.807e-06 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663522+0000", 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4786999999999999e-05 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663533+0000", 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0974e-05 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663597+0000", 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4194000000000002e-05 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.663608+0000", 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1505000000000001e-05 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823320+0000", 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.15971187000000001 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823457+0000", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013665199999999999 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.823613+0000", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00015588599999999999 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.824471+0000", 2024-05-09T08:58:00.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085837200000000004 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826477+0000", 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020054309999999998 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826518+0000", 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1449000000000002e-05 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:57.826561+0000", 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2703000000000001e-05 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043569675, 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073092820000000003, 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.109 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 108 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056153+0000", 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056154+0000", 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3099999999999996e-07 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056162+0000", 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1790000000000002e-06 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056277+0000", 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000115016 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056283+0000", 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8519999999999998e-06 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058631+0000", 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023476170000000002 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058678+0000", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7058999999999998e-05 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.058753+0000", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5109000000000002e-05 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063309+0000", 2024-05-09T08:58:00.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045561059999999999 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063378+0000", 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8429000000000005e-05 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063423+0000", 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5507000000000003e-05 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063463+0000", 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9477000000000002e-05 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043522083, 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073512789999999996, 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 109 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056201+0000", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056202+0000", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2399999999999997e-07 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056247+0000", 2024-05-09T08:58:00.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4864999999999998e-05 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056304+0000", 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7024999999999997e-05 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056309+0000", 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6330000000000001e-06 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059148+0000", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028390440000000002 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059213+0000", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4708000000000005e-05 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.059286+0000", 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3462999999999998e-05 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063363+0000", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0040768740000000003 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063473+0000", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000109447 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063512+0000", 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9628000000000001e-05 2024-05-09T08:58:00.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063552+0000", 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9867999999999998e-05 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043472216, 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.008543854, 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 110 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056251+0000", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056252+0000", 2024-05-09T08:58:00.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2900000000000003e-07 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056255+0000", 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.213e-06 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056326+0000", 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0792999999999998e-05 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056336+0000", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0081999999999999e-05 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062770+0000", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0064343170000000002 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062848+0000", 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8023999999999997e-05 2024-05-09T08:58:00.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.062981+0000", 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000133557 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064645+0000", 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001663042 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064711+0000", 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6346999999999993e-05 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064756+0000", 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4749000000000001e-05 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064795+0000", 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9100999999999997e-05 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043466416, 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011100085000000001, 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 111 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056257+0000", 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1600000000000002e-07 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056260+0000", 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5500000000000001e-06 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056348+0000", 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8528000000000003e-05 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056353+0000", 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7709999999999997e-06 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063600+0000", 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072473670000000002 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063657+0000", 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6861999999999999e-05 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.063733+0000", 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6027999999999998e-05 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064698+0000", 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00096459900000000001 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067273+0000", 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0025749280000000002 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067319+0000", 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6295000000000003e-05 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067357+0000", 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7840999999999999e-05 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043354616, 2024-05-09T08:58:00.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011072040999999999, 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 112 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056368+0000", 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056369+0000", 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7499999999999999e-07 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056461+0000", 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1978999999999995e-05 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056488+0000", 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6483e-05 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056493+0000", 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0749999999999997e-06 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064013+0000", 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075198169999999998 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064089+0000", 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6674000000000005e-05 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064209+0000", 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119715 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065714+0000", 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0015054199999999999 2024-05-09T08:58:00.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067365+0000", 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016509280000000001 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067406+0000", 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0606999999999999e-05 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067441+0000", 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4567999999999998e-05 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043257454, 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011052008, 2024-05-09T08:58:00.144 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 113 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056466+0000", 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3099999999999997e-07 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056469+0000", 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.219e-06 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056506+0000", 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6081999999999998e-05 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056516+0000", 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0117e-05 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064825+0000", 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083091769999999992 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064879+0000", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4302999999999998e-05 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.064951+0000", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1923999999999998e-05 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065764+0000", 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00081261899999999999 2024-05-09T08:58:00.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067447+0000", 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016831389999999999 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067486+0000", 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8829999999999999e-05 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067518+0000", 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1967000000000001e-05 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.151 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043251696, 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011124497000000001, 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 114 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056471+0000", 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056472+0000", 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5600000000000002e-07 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056475+0000", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8509999999999999e-06 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056532+0000", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.7398000000000002e-05 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056535+0000", 2024-05-09T08:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.546e-06 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065185+0000", 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0086491880000000004 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065259+0000", 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4011999999999995e-05 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065397+0000", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000138305 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066837+0000", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014403109999999999 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067525+0000", 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068810300000000002 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067563+0000", 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7973999999999999e-05 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067596+0000", 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2552999999999998e-05 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.157 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043178056, 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.011171238, 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056545+0000", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056546+0000", 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5199999999999997e-07 2024-05-09T08:58:00.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056595+0000", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9116999999999997e-05 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056616+0000", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1464999999999999e-05 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056620+0000", 2024-05-09T08:58:00.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.348e-06 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065784+0000", 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091640449999999991 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065826+0000", 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2920999999999998e-05 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.065905+0000", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8114000000000005e-05 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067603+0000", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016986 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067657+0000", 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3516000000000001e-05 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067691+0000", 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4473e-05 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067716+0000", 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5086999999999999e-05 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.163 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.04312624400001, 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013023677000000001, 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056597+0000", 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7e-07 2024-05-09T08:58:00.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056599+0000", 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2189999999999998e-06 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056630+0000", 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0209000000000002e-05 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056632+0000", 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6230000000000001e-06 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066303+0000", 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096707070000000006 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066392+0000", 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.9321999999999996e-05 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066512+0000", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119568 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067764+0000", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001252527 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069532+0000", 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0017678870000000001 2024-05-09T08:58:00.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069583+0000", 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1119000000000003e-05 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.069621+0000", 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7126000000000003e-05 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 122.043122525, 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.015821907999999999, 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-09T08:58:00.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056601+0000", 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.36e-07 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056603+0000", 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9269999999999999e-06 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056638+0000", 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5015000000000003e-05 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.056641+0000", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066865+0000", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010223833 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.066957+0000", 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1630000000000002e-05 2024-05-09T08:58:00.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067082+0000", 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012561300000000001 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.067797+0000", 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00071505400000000002 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072333+0000", 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045354410000000003 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072382+0000", 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9582000000000001e-05 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.072422+0000", 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0009000000000003e-05 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 121.558066068, 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0075546349999999996, 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.176 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541657+0000", 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541658+0000", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999997e-07 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541661+0000", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1200000000000002e-06 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541738+0000", 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7545000000000004e-05 2024-05-09T08:58:00.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541741+0000", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4899999999999999e-06 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547460+0000", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057187749999999997 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547506+0000", 2024-05-09T08:58:00.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5917000000000001e-05 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547576+0000", 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0104000000000005e-05 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549118+0000", 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001542082 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549146+0000", 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8056999999999999e-05 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549174+0000", 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.783e-05 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549212+0000", 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8077000000000002e-05 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.181 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 121.558049584, 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076844410000000002, 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541674+0000", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6300000000000001e-07 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541679+0000", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6169999999999998e-06 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541746+0000", 2024-05-09T08:58:00.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6865000000000004e-05 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541749+0000", 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.446e-06 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547833+0000", 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060837920000000002 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547884+0000", 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1103000000000002e-05 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.547952+0000", 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8099e-05 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549261+0000", 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001308732 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549273+0000", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1985e-05 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549320+0000", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7494999999999998e-05 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549358+0000", 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7944000000000001e-05 2024-05-09T08:58:00.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 121.557967627, 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077795629999999998, 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541755+0000", 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541756+0000", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.91e-07 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541801+0000", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5293000000000002e-05 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541833+0000", 2024-05-09T08:58:00.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1989000000000002e-05 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541836+0000", 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.281e-06 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548191+0000", 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0063550339999999999 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548235+0000", 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3640000000000002e-05 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548300+0000", 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4916e-05 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549368+0000", 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010684799999999999 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549453+0000", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4363000000000005e-05 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549500+0000", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.7540999999999998e-05 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549535+0000", 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4734999999999999e-05 2024-05-09T08:58:00.193 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4216.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 121.557920067, 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0079419810000000007, 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-09T08:58:00.194 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4216", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.183:0/724028894", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:58:00.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541803+0000", 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541805+0000", 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.917e-06 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541845+0000", 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9203999999999999e-05 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.541847+0000", 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4679999999999999e-06 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548814+0000", 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0069672689999999999 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548858+0000", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3630000000000001e-05 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.548930+0000", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2268999999999995e-05 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549548+0000", 2024-05-09T08:58:00.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00061749799999999996 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549683+0000", 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00013549699999999999 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549710+0000", 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7342999999999999e-05 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:55:58.549745+0000", 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4513e-05 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.3.13:26 2.1d 2:b94f8861:::mds3_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 96.354291622999995, 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071784839999999997, 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.200 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745431+0000", 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745432+0000", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4699999999999998e-07 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745435+0000", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7079999999999999e-06 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745606+0000", 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000170168 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.745610+0000", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1930000000000003e-06 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747795+0000", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021848610000000002 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.747914+0000", 2024-05-09T08:58:00.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000118936 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.748021+0000", 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010761599999999999 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.749627+0000", 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001606026 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752488+0000", 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028609149999999999 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752563+0000", 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5215000000000004e-05 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:23.752610+0000", 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6598999999999999e-05 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.205 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:16 2.1d 2:b9436051:::62e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.338212178000006, 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0071432029999999999, 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761511+0000", 2024-05-09T08:58:00.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2200000000000004e-07 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761514+0000", 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.531e-06 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761864+0000", 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00035041199999999998 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761872+0000", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5259999999999996e-06 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762388+0000", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00051647599999999998 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762464+0000", 2024-05-09T08:58:00.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.5294000000000001e-05 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762588+0000", 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012400200000000001 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.766375+0000", 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0037876780000000001 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768556+0000", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0021810039999999998 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768610+0000", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3440999999999999e-05 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768654+0000", 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4317000000000003e-05 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.211 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:21 2.1d 2:be329c1b:::631.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.338207862000004, 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0073061200000000001, 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761515+0000", 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3900000000000001e-07 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761518+0000", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.294e-06 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761901+0000", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00038311299999999998 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761907+0000", 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9519999999999999e-06 2024-05-09T08:58:00.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762824+0000", 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091678799999999998 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.762938+0000", 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000114838 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:00.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763067+0000", 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000128097 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.767937+0000", 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0048708509999999998 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768712+0000", 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077438799999999999 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768782+0000", 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0301000000000003e-05 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768821+0000", 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9258999999999998e-05 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:22 2.1d 2:be329c1b:::631.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.338198899000005, 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007438208, 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761524+0000", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6600000000000003e-07 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761527+0000", 2024-05-09T08:58:00.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5809999999999999e-06 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761918+0000", 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00039088399999999998 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761924+0000", 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3790000000000003e-06 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763901+0000", 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019763519999999998 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.763989+0000", 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7850999999999996e-05 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764117+0000", 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012835099999999999 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768058+0000", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0039416310000000001 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768870+0000", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000811242 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768929+0000", 2024-05-09T08:58:00.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.9185999999999999e-05 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768962+0000", 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3485000000000002e-05 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(mds.4.15:23 2.1c 2:399c8d31:::104.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:58:00.223 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 90.338194838999996, 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007064046, 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:58:00.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761528+0000", 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761535+0000", 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6540000000000002e-06 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-09T08:58:00.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761935+0000", 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00040023100000000002 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.761940+0000", 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4680000000000003e-06 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764803+0000", 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00286294 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.764948+0000", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00014544699999999999 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.765181+0000", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000233017 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768031+0000", 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0028492790000000001 2024-05-09T08:58:00.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768507+0000", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00047597900000000002 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768557+0000", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.041e-05 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-09T08:56:29.768592+0000", 2024-05-09T08:58:00.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5380999999999999e-05 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-09T08:58:00.229 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-09T08:58:00.230 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:58:00.232+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:58:00.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-09T08:58:00.331 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:58:00.332+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:58:00.346 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:00.346 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:00.346 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:58:00.346 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:00.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-09T08:58:00.432 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:58:00.433+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:58:00.465 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:00.466 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:00.466 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:58:00.466 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:58:00.466 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:00.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-09T08:58:00.533 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:58:00.534+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.905717459, 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00531023, 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680311+0000", 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680312+0000", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7199999999999999e-07 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680315+0000", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680449+0000", 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013403699999999999 2024-05-09T08:58:00.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680456+0000", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5899999999999996e-06 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681503+0000", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010475809999999999 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681586+0000", 2024-05-09T08:58:00.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2788999999999996e-05 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681701+0000", 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011441499999999999 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684866+0000", 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031656089999999998 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685528+0000", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066194799999999999 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685581+0000", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3035000000000003e-05 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685621+0000", 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0176999999999998e-05 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.591 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.90571189000001, 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046284120000000002, 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680317+0000", 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7300000000000002e-07 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680346+0000", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8951000000000001e-05 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680480+0000", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013432400000000001 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680489+0000", 2024-05-09T08:58:00.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2789999999999998e-06 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680549+0000", 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0458999999999998e-05 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680703+0000", 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015394900000000001 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680957+0000", 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025449799999999997 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683761+0000", 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028037370000000002 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684844+0000", 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001083281 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684896+0000", 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1779e-05 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684945+0000", 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8881999999999997e-05 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.597 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.905680202, 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060001129999999996, 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680348+0000", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680349+0000", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5499999999999998e-07 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680352+0000", 2024-05-09T08:58:00.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7860000000000001e-06 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680514+0000", 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000161877 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680537+0000", 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3652e-05 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682337+0000", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018002300000000001 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682448+0000", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110071 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682576+0000", 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012847499999999999 2024-05-09T08:58:00.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685934+0000", 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033575300000000001 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686254+0000", 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00032031600000000001 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686305+0000", 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1295999999999998e-05 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686348+0000", 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3325000000000003e-05 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:00.603 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.905675516, 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046943519999999997, 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680353+0000", 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8599999999999999e-07 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680356+0000", 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2369999999999999e-06 2024-05-09T08:58:00.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680573+0000", 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021728200000000001 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680581+0000", 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2640000000000008e-06 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681577+0000", 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099568500000000006 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681695+0000", 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011858000000000001 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681841+0000", 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014513199999999999 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684399+0000", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0025589179999999999 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684955+0000", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055584299999999998 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685009+0000", 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4148999999999999e-05 2024-05-09T08:58:00.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685047+0000", 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7975999999999997e-05 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.905671688, 2024-05-09T08:58:00.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060845600000000001, 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:58:00.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680357+0000", 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6100000000000002e-07 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680363+0000", 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8089999999999998e-06 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680599+0000", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023619299999999999 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682899+0000", 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022938379999999999 2024-05-09T08:58:00.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683027+0000", 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012804400000000001 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686441+0000", 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003414425 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.0.4:2 2.f e20/17 2:f1f69726:::400.00000000:head v 20'1, mlcod=0'0)", 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 113.904788822, 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050033949999999999, 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681240+0000", 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681241+0000", 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3060000000000001e-06 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681251+0000", 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0414000000000001e-05 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681777+0000", 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00052538599999999997 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681788+0000", 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1467e-05 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682015+0000", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000226584 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682034+0000", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-05 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686190+0000", 2024-05-09T08:58:00.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041555159999999997 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686243+0000", 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3631999999999997e-05 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:30 2.13 2:ca4dd7f5:::611.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:58:00.618 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.859215937, 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060619080000000004, 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726813+0000", 2024-05-09T08:58:00.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726814+0000", 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.201e-06 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726820+0000", 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726838+0000", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8142999999999999e-05 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726846+0000", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.255e-06 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726900+0000", 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4564000000000001e-05 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.726979+0000", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8984000000000001e-05 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727086+0000", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010674 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728358+0000", 2024-05-09T08:58:00.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001272193 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732779+0000", 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044212339999999996 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732833+0000", 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3457e-05 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732875+0000", 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1755000000000003e-05 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(mds.1.9:32 2.8 e20/17 2:1470b2f8:::613.00000000:head v 20'2, mlcod=0'0)", 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 108.858543162, 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0054414570000000002, 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727485+0000", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727486+0000", 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.58e-07 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727492+0000", 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6949999999999997e-06 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727500+0000", 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1159999999999999e-06 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727506+0000", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8780000000000003e-06 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727552+0000", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6440999999999998e-05 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.727571+0000", 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8887999999999999e-05 2024-05-09T08:58:00.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732895+0000", 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053244140000000004 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.732927+0000", 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1566999999999998e-05 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.855822495, 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057127890000000002, 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730206+0000", 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.77e-07 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730209+0000", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730281+0000", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2059999999999998e-05 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730286+0000", 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9690000000000002e-06 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731025+0000", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073917999999999998 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731123+0000", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8245000000000006e-05 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731251+0000", 2024-05-09T08:58:00.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012736299999999999 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733466+0000", 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00221535 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735827+0000", 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023607559999999999 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735879+0000", 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2231000000000002e-05 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735919+0000", 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9669000000000003e-05 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.634 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.855814454, 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056884379999999997, 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730214+0000", 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.79e-07 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730217+0000", 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8550000000000002e-06 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730322+0000", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104808 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730327+0000", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7140000000000001e-06 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732700+0000", 2024-05-09T08:58:00.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023734680000000001 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732767+0000", 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6878000000000006e-05 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732894+0000", 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000126666 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.734124+0000", 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001230047 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735807+0000", 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001683103 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735860+0000", 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2599000000000002e-05 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.735903+0000", 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3021e-05 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.855810134, 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083886859999999994, 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730218+0000", 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730219+0000", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6399999999999998e-07 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730221+0000", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730349+0000", 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127627 2024-05-09T08:58:00.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730353+0000", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1690000000000002e-06 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731500+0000", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001147159 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731620+0000", 2024-05-09T08:58:00.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000119778 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731733+0000", 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011291299999999999 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733471+0000", 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017383279999999999 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738543+0000", 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0050718330000000004 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738583+0000", 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9870999999999998e-05 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738607+0000", 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4088999999999999e-05 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.645 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.855805829, 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085381080000000008, 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.646 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730223+0000", 2024-05-09T08:58:00.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-07 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730225+0000", 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.844e-06 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730363+0000", 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013846299999999999 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730368+0000", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1749999999999998e-06 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731966+0000", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001598478 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732040+0000", 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4381000000000005e-05 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732151+0000", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110519 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733476+0000", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001324661 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738662+0000", 2024-05-09T08:58:00.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051863569999999999 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738719+0000", 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6975999999999998e-05 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738761+0000", 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1961e-05 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.85580258100001, 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086061690000000003, 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:58:00.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730226+0000", 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1100000000000002e-07 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730228+0000", 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.612e-06 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730377+0000", 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000149297 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.730381+0000", 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732413+0000", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020319750000000001 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732523+0000", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110029 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738832+0000", 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063089549999999998 2024-05-09T08:58:00.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:11 2.1f 2:ffe2d39d:::622.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 96.840730207999997, 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047747429999999997, 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745298+0000", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745300+0000", 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.508e-06 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745307+0000", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6429999999999998e-06 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7484e-05 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745343+0000", 2024-05-09T08:58:00.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9830000000000002e-06 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745412+0000", 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9385000000000001e-05 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745537+0000", 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012438799999999999 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745654+0000", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117562 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746864+0000", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012098759999999999 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750017+0000", 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003153023 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750047+0000", 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0266000000000001e-05 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750073+0000", 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5624999999999999e-05 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 96.840718960999993, 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048592749999999997, 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7300000000000002e-07 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745312+0000", 2024-05-09T08:58:00.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745367+0000", 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4580999999999998e-05 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745372+0000", 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4179999999999996e-06 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745903+0000", 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00053062300000000003 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745981+0000", 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7540000000000006e-05 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746106+0000", 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125344 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747268+0000", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011616389999999999 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750110+0000", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028420260000000001 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750142+0000", 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2305000000000003e-05 2024-05-09T08:58:00.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750169+0000", 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6937e-05 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 96.840714539999993, 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059131039999999998, 2024-05-09T08:58:00.667 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745314+0000", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745316+0000", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.885e-06 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745386+0000", 2024-05-09T08:58:00.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9766999999999998e-05 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745939+0000", 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00055316499999999999 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746024+0000", 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4858000000000006e-05 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746155+0000", 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000130569 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746276+0000", 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120927 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747528+0000", 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012528019999999999 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751107+0000", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035790990000000001 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751173+0000", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5155999999999998e-05 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751227+0000", 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4487999999999997e-05 2024-05-09T08:58:00.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:13 2.13 2:c81eaac8:::62d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.824610574999994, 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047594769999999998, 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761418+0000", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761420+0000", 2024-05-09T08:58:00.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.668e-06 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761426+0000", 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761465+0000", 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8568000000000001e-05 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761474+0000", 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2590000000000006e-06 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761541+0000", 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6872999999999995e-05 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761655+0000", 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000113448 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761736+0000", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1317999999999996e-05 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762413+0000", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067763199999999995 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766071+0000", 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036576400000000002 2024-05-09T08:58:00.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766125+0000", 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4138999999999998e-05 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766177+0000", 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2237999999999997e-05 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:14 2.13 2:c81eaac8:::62d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.824599243999998, 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0056428299999999997, 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761429+0000", 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761430+0000", 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8799999999999998e-07 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761432+0000", 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.649e-06 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761502+0000", 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9284999999999998e-05 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761507+0000", 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.412e-06 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761951+0000", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000444079 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762036+0000", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5259999999999993e-05 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762183+0000", 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000146611 2024-05-09T08:58:00.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.763294+0000", 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001111017 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766968+0000", 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036740789999999998 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767039+0000", 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1099000000000005e-05 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767072+0000", 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2950999999999997e-05 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:1 2.1f 2:fb684bcd:::204.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:58:00.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.824591294000001, 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071013400000000003, 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761437+0000", 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761438+0000", 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3999999999999997e-07 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761442+0000", 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.279e-06 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761549+0000", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010744899999999999 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761557+0000", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6569999999999997e-06 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761609+0000", 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2175000000000001e-05 2024-05-09T08:58:00.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761735+0000", 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012618300000000001 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761865+0000", 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000129696 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:00.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762676+0000", 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081115600000000005 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768431+0000", 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057543999999999998 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768495+0000", 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4574999999999998e-05 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768539+0000", 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3430000000000003e-05 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:31 2.1f 2:ff4f7703:::204.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.824585416000005, 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071841070000000003, 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:00.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761443+0000", 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761444+0000", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9000000000000002e-07 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761446+0000", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6010000000000002e-06 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761578+0000", 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131738 2024-05-09T08:58:00.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.761583+0000", 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2850000000000002e-06 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762260+0000", 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00067676200000000002 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.762361+0000", 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010082900000000001 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.768627+0000", 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062665020000000002 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:00.694 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:00.695 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:00.696 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:58:00.697+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:58:00.696 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-09T08:58:00.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.736+0000 7fcf82ffd640 1 -- 172.21.15.49:0/2100973065 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v58) ==== 91+0+0 (secure 0 0 0) 0x7fcf8405cb20 con 0x7fcf8c0c6b80 2024-05-09T08:58:00.736 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:58:00.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.738+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 msgr2=0x7fcf6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.738+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 0x7fcf6c06e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0d0110 tx=0x7fcf74009190 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 msgr2=0x7fcf8c1f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400c3f0 tx=0x7fcf8400c8b0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 shutdown_connections 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf6c06be30 0x7fcf6c06e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf8c154f70 0x7fcf8c0b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf8c154b90 0x7fcf8c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 --2- 172.21.15.49:0/2100973065 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf8c0c6b80 0x7fcf8c1f4de0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 >> 172.21.15.49:0/2100973065 conn(0x7fcf8c074840 msgr2=0x7fcf8c158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 shutdown_connections 2024-05-09T08:58:00.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.739+0000 7fcf90bb0640 1 -- 172.21.15.49:0/2100973065 wait complete. 2024-05-09T08:58:00.797 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:58:00.798+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:58:00.808 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:00.809 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:00.809 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:58:00.809 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:00.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-09T08:58:00.854 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-05-09T08:58:00.854 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-09T08:58:00.854 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-09T08:58:00.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-05-09T08:58:00.898 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-09T08:58:00.900+0000 7f22c34fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36596) UID: 0 2024-05-09T08:58:00.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.965+0000 7f443799e640 1 Processor -- start 2024-05-09T08:58:00.964 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.965+0000 7f443799e640 1 -- start start 2024-05-09T08:58:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f443799e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4430154f70 0x7f44300c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f443799e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44300c6fd0 con 0x7f4430154b90 2024-05-09T08:58:00.965 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f443799e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4430072600 con 0x7f44300c6b80 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f443799e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f44300728e0 con 0x7f4430154f70 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4430154f70 0x7f44300c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4430154f70 0x7f44300c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53868/0 (socket says 172.21.15.49:53868) 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 -- 172.21.15.49:0/4234152659 learned_addr learned my addr 172.21.15.49:0/4234152659 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f443573a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4430154b90 0x7f44300cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49486/0 (socket says 172.21.15.49:49486) 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 -- 172.21.15.49:0/4234152659 >> v1:172.21.15.183:6789/0 conn(0x7f44300c6b80 legacy=0x7f4430071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 -- 172.21.15.49:0/4234152659 >> v1:172.21.15.49:6789/0 conn(0x7f4430154b90 legacy=0x7f44300cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.966+0000 7f4434f39640 1 -- 172.21.15.49:0/4234152659 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44300c5880 con 0x7f4430154f70 2024-05-09T08:58:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.967+0000 7f4434f39640 1 --2- 172.21.15.49:0/4234152659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154f70 0x7f44300c6640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4420009870 tx=0x7f442002a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b46266d481ee59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.967+0000 7f4427fff640 1 -- 172.21.15.49:0/4234152659 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420037070 con 0x7f4430154f70 2024-05-09T08:58:00.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.967+0000 7f4427fff640 1 -- 172.21.15.49:0/4234152659 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420032640 con 0x7f4430154f70 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.967+0000 7f4427fff640 1 -- 172.21.15.49:0/4234152659 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420032a10 con 0x7f4430154f70 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 -- 172.21.15.49:0/4234152659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154f70 msgr2=0x7f44300c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 --2- 172.21.15.49:0/4234152659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154f70 0x7f44300c6640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4420009870 tx=0x7f442002a3c0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 -- 172.21.15.49:0/4234152659 shutdown_connections 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 --2- 172.21.15.49:0/4234152659 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154f70 0x7f44300c6640 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 -- 172.21.15.49:0/4234152659 >> 172.21.15.49:0/4234152659 conn(0x7f4430074840 msgr2=0x7f4430074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 -- 172.21.15.49:0/4234152659 shutdown_connections 2024-05-09T08:58:00.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.968+0000 7f443799e640 1 -- 172.21.15.49:0/4234152659 wait complete. 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.969+0000 7f443799e640 1 Processor -- start 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.969+0000 7f443799e640 1 -- start start 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 0x7f44300b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 0x7f443014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:00.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f44300c6120 con 0x7f4430154b90 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44300c5e40 con 0x7f4430154f70 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443799e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f443015af40 con 0x7f44300c6b80 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:58:00.970 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53870/0 (socket says 172.21.15.49:53870) 2024-05-09T08:58:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 -- 172.21.15.49:0/2550029301 learned_addr learned my addr 172.21.15.49:0/2550029301 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443573a640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 0x7f44300b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:00.972 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4435f3b640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 0x7f443014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 -- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 msgr2=0x7f44300b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 0x7f44300b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 -- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 msgr2=0x7f443014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 0x7f443014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 -- 172.21.15.49:0/2550029301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44301f2320 con 0x7f4430154b90 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f443573a640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 0x7f44300b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4435f3b640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 0x7f443014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.970+0000 7f4434f39640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4420000c00 tx=0x7f44200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.971+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420037040 con 0x7f4430154b90 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.971+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420042070 con 0x7f4430154b90 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.971+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443014bda0 con 0x7f4430154b90 2024-05-09T08:58:00.973 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.971+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420030bd0 con 0x7f4430154b90 2024-05-09T08:58:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.971+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4430153fb0 con 0x7f4430154b90 2024-05-09T08:58:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.972+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4430159960 con 0x7f4430154b90 2024-05-09T08:58:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.972+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4420032500 con 0x7f4430154b90 2024-05-09T08:58:00.974 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.973+0000 7f4425ffb640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 0x7f441406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.973+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5886+0+0 (secure 0 0 0) 0x7f44200aba70 con 0x7f4430154b90 2024-05-09T08:58:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.974+0000 7f443573a640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 0x7f441406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:00.975 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.974+0000 7f443573a640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 0x7f441406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f442c000f60 tx=0x7f442c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:00.977 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:00.978+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4430159960 con 0x7f4430154b90 2024-05-09T08:58:00.979 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-09T08:58:00.999 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:58:01.000+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 114.41150967999999, 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046547589999999996, 2024-05-09T08:58:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680575+0000", 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680586+0000", 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0739e-05 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680596+0000", 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0030999999999999e-05 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680646+0000", 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0164999999999998e-05 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680658+0000", 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1592e-05 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680766+0000", 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010822100000000001 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680908+0000", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000142304 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681049+0000", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014077600000000001 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682823+0000", 2024-05-09T08:58:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017734859999999999 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685114+0000", 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00229137 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685167+0000", 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3235000000000001e-05 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.685230+0000", 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000001e-05 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 114.411485328, 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064196180000000002, 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680600+0000", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8800000000000003e-07 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680604+0000", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7809999999999999e-06 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681337+0000", 2024-05-09T08:58:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000733342 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681345+0000", 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4730000000000003e-06 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681402+0000", 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6898000000000003e-05 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681492+0000", 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0229000000000006e-05 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681627+0000", 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013552799999999999 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682829+0000", 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00120118 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686936+0000", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041069820000000003 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686991+0000", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105000000000002e-05 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687019+0000", 2024-05-09T08:58:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8612e-05 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 114.41147957299999, 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065931649999999998, 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680605+0000", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680606+0000", 2024-05-09T08:58:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7099999999999997e-07 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680619+0000", 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3199e-05 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681876+0000", 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001256879 2024-05-09T08:58:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.681883+0000", 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.306e-06 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682162+0000", 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027878400000000002 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682288+0000", 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125618 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682426+0000", 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013802000000000001 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683439+0000", 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010131070000000001 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687077+0000", 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036383700000000001 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687155+0000", 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8208999999999996e-05 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.687199+0000", 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3302e-05 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e19)", 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 114.411464342, 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061876589999999999, 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680621+0000", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4700000000000002e-07 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.680623+0000", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.424e-06 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682851+0000", 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022271439999999999 2024-05-09T08:58:01.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682859+0000", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5790000000000004e-06 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.682914+0000", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5133000000000002e-05 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683022+0000", 2024-05-09T08:58:01.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000107847 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 4", 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.683130+0000", 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010812800000000001 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.684041+0000", 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091058800000000004 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686707+0000", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026662629999999999 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686755+0000", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7880000000000002e-05 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:06.686808+0000", 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3226000000000002e-05 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.115 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:13 2.15 2:aa0f8bf4:::101.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 109.36812635699999, 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007504943, 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723959+0000", 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2999999999999999e-07 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723964+0000", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6129999999999997e-06 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723995+0000", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1359000000000002e-05 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724014+0000", 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8369e-05 2024-05-09T08:58:01.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724830+0000", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000816076 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724939+0000", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109211 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.725054+0000", 2024-05-09T08:58:01.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114459 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.728186+0000", 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031323050000000002 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731384+0000", 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031981380000000001 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731425+0000", 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1088999999999999e-05 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731464+0000", 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8494000000000001e-05 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.1.9:16 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.121 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 109.368118396, 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073941199999999997, 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723967+0000", 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1200000000000003e-07 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.723974+0000", 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2620000000000001e-06 2024-05-09T08:58:01.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724039+0000", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4412000000000006e-05 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724045+0000", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.835e-06 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724242+0000", 2024-05-09T08:58:01.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019695300000000001 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724352+0000", 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110615 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.724479+0000", 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012685800000000001 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.730496+0000", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060167839999999998 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731266+0000", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077040099999999999 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731314+0000", 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7808e-05 2024-05-09T08:58:01.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:11.731361+0000", 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.668e-05 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.360910367, 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069294350000000003, 2024-05-09T08:58:01.127 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731175+0000", 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731176+0000", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.466e-06 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731183+0000", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5039999999999999e-06 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731208+0000", 2024-05-09T08:58:01.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5145000000000001e-05 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731216+0000", 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7420000000000003e-06 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731329+0000", 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011284400000000001 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731446+0000", 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011633099999999999 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731561+0000", 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115846 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733890+0000", 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023281479999999999 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738041+0000", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041517489999999997 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738076+0000", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4251000000000001e-05 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738104+0000", 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8408999999999999e-05 2024-05-09T08:58:01.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.2.11:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.36089962200001, 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007071735, 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731185+0000", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731186+0000", 2024-05-09T08:58:01.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.89e-07 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731188+0000", 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.785e-06 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731247+0000", 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8164000000000001e-05 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731253+0000", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9549999999999997e-06 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731899+0000", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064632199999999996 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.731977+0000", 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8436000000000006e-05 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.732101+0000", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000123325 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.733895+0000", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001794213 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738154+0000", 2024-05-09T08:58:01.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042587989999999997 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738214+0000", 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9954000000000002e-05 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:17.738257+0000", 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3393000000000003e-05 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:5 2.9 2:941d59bc:::61f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346925049000006, 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003927013, 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:58:01.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745160+0000", 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745161+0000", 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.339e-06 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745168+0000", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745228+0000", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0776000000000002e-05 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745236+0000", 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6760000000000007e-06 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0210999999999997e-05 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745428+0000", 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000131299 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:01.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745556+0000", 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012859300000000001 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747397+0000", 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018411 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.748978+0000", 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015809019999999999 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749044+0000", 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5680999999999997e-05 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749087+0000", 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3093999999999998e-05 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.144 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346914940000005, 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0044315149999999996, 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745170+0000", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.17e-07 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745172+0000", 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.672e-06 2024-05-09T08:58:01.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745256+0000", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3899000000000004e-05 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745259+0000", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.236e-06 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745935+0000", 2024-05-09T08:58:01.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000676327 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746007+0000", 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2006000000000001e-05 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746408+0000", 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000400388 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747464+0000", 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001056614 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749534+0000", 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020692649999999998 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749564+0000", 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0802000000000001e-05 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.749602+0000", 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.150 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:7 2.6 2:6642f7a8:::620.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346912302000007, 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006199179, 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745173+0000", 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0200000000000001e-07 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745174+0000", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2079999999999999e-06 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745292+0000", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117581 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745296+0000", 2024-05-09T08:58:01.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0339999999999998e-06 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745340+0000", 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4501e-05 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745457+0000", 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116869 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745567+0000", 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010974199999999999 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747425+0000", 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018578220000000001 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751301+0000", 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038762290000000001 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751349+0000", 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7787000000000002e-05 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751372+0000", 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3204e-05 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.156 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346896700000002, 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006322215, 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745188+0000", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4999999999999999e-07 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745190+0000", 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6589999999999999e-06 2024-05-09T08:58:01.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745308+0000", 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011752300000000001 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745310+0000", 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745926+0000", 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061598999999999996 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746006+0000", 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9493999999999995e-05 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746144+0000", 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013828399999999999 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747452+0000", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013084469999999999 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751421+0000", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039684899999999999 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751484+0000", 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3293999999999994e-05 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751510+0000", 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6324000000000001e-05 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346894051000007, 2024-05-09T08:58:01.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00639333, 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745191+0000", 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999998e-07 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745202+0000", 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0559e-05 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745334+0000", 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013185000000000001 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745337+0000", 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.977e-06 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746369+0000", 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010325040000000001 2024-05-09T08:58:01.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746475+0000", 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105993 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746594+0000", 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000118696 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747496+0000", 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090186900000000002 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751539+0000", 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040435979999999998 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751565+0000", 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5816000000000002e-05 2024-05-09T08:58:01.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751584+0000", 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9228e-05 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346879924999996, 2024-05-09T08:58:01.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0064964159999999996, 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:58:01.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745205+0000", 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.11e-07 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745349+0000", 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000141917 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745352+0000", 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746876+0000", 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015240270000000001 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.746946+0000", 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9645000000000002e-05 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 6", 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747088+0000", 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014231899999999999 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747635+0000", 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054701000000000001 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751620+0000", 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003985036 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751666+0000", 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5904e-05 2024-05-09T08:58:01.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.751701+0000", 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5135999999999997e-05 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:25 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.174 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346877645999996, 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0057329700000000004, 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745207+0000", 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9999999999999999e-07 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745212+0000", 2024-05-09T08:58:01.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7219999999999999e-06 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745360+0000", 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000147851 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745364+0000", 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3270000000000002e-06 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745420+0000", 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5166e-05 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745524+0000", 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010464800000000001 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745724+0000", 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019941800000000001 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.747438+0000", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017147359999999999 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750875+0000", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034365149999999998 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750913+0000", 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8073e-05 2024-05-09T08:58:01.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750940+0000", 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7314e-05 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:31 2.d 2:b73edb83:::203.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346869741000006, 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053075229999999998, 2024-05-09T08:58:01.180 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745215+0000", 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0800000000000001e-07 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745216+0000", 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0999999999999997e-07 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745429+0000", 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00021320200000000001 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745436+0000", 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5230000000000001e-06 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745531+0000", 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5346000000000005e-05 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745626+0000", 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4506999999999995e-05 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.750523+0000", 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048969269999999997 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.184 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.3.13:33 2.a 2:556620e9:::203.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.346868220000005, 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98420219499999995, 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745217+0000", 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.98e-07 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:23.745218+0000", 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9300000000000006e-07 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728038+0000", 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.98281996000000005 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728056+0000", 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7846000000000002e-05 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728209+0000", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015360100000000001 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.728322+0000", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011277699999999999 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:24.729419+0000", 2024-05-09T08:58:01.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010968200000000001 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:9 2.9 2:954a9a2b:::62b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.326792370999996, 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039188570000000004, 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765293+0000", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765294+0000", 2024-05-09T08:58:01.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5570000000000001e-06 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765301+0000", 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4930000000000003e-06 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765339+0000", 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8603000000000003e-05 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765346+0000", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0690000000000004e-06 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765427+0000", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0479e-05 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765565+0000", 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013816100000000001 2024-05-09T08:58:01.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765677+0000", 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111706 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766863+0000", 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011859539999999999 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769158+0000", 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002295624 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769187+0000", 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9062000000000001e-05 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769211+0000", 2024-05-09T08:58:01.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4148999999999999e-05 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.326770910999997, 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040616020000000001, 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765314+0000", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765315+0000", 2024-05-09T08:58:01.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7399999999999998e-07 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765320+0000", 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4620000000000004e-06 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765394+0000", 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4197999999999997e-05 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765413+0000", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8389e-05 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766373+0000", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096058700000000005 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766445+0000", 2024-05-09T08:58:01.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2317000000000006e-05 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.766622+0000", 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000176742 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.767185+0000", 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000562957 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769265+0000", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00207982 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769332+0000", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6544000000000005e-05 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769376+0000", 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4112e-05 2024-05-09T08:58:01.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(mds.4.15:29 2.d 2:b2057508:::204.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e20)", 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 91.326763905999996, 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037006019999999999, 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "continuous": false, 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-05-09T08:58:01.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765321+0000", 2024-05-09T08:58:01.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9900000000000002e-07 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765322+0000", 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.062e-06 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765433+0000", 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110216 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-05-09T08:58:01.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765440+0000", 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0829999999999998e-06 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765497+0000", 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7534999999999997e-05 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.765601+0000", 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103667 2024-05-09T08:58:01.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-05-09T08:56:29.769022+0000", 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0034207399999999998 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-05-09T08:58:01.205 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:01.207 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-09T08:58:01.207 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.174+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd out", "ids": ["0"]} v 0) -- 0x7f443015bae0 con 0x7f4430154b90 2024-05-09T08:58:01.208 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-05-09T08:58:01.174+0000 7fbf24104640 -1 mon.a@0(leader).osd e58 definitely_dead 0 2024-05-09T08:58:01.208 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:58:01.208+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:58:01.307 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:58:01.308+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:58:01.358 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:01.358 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:01.358 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:58:01.358 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:01.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-09T08:58:01.378 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-09T08:58:01.408 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:58:01.409+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:58:01.479 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T08:58:01.479 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T08:58:01.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.511+0000 7f47c2bc1640 1 Processor -- start 2024-05-09T08:58:01.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.511+0000 7f47c2bc1640 1 -- start start 2024-05-09T08:58:01.512 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-05-09T08:58:01.511+0000 7f03089d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 35647) UID: 0 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c2bc1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47bc0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c2bc1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc072600 con 0x7f47bc0c6d50 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c2bc1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc0728e0 con 0x7f47bc154f70 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c2bc1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc0c5880 con 0x7f47bc154b90 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c115e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f47bc0c6d50 0x7f47bc071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49494/0 (socket says 172.21.15.49:49494) 2024-05-09T08:58:01.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c115e640 1 -- 172.21.15.49:0/3630229074 learned_addr learned my addr 172.21.15.49:0/3630229074 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47c095d640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47bc0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548764719 0 0) 0x7f47bc072600 con 0x7f47bc0c6d50 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.512+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47ac003520 con 0x7f47bc0c6d50 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2831861877 0 0) 0x7f47ac003520 con 0x7f47bc0c6d50 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 msgr2=0x7f47bc0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:01.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47bc0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 >> v1:172.21.15.183:6789/0 conn(0x7f47bc154f70 legacy=0x7f47bc0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0c5b60 con 0x7f47bc0c6d50 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f47b40019f0 con 0x7f47bc0c6d50 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 >> v1:172.21.15.49:6789/0 conn(0x7f47bc0c6d50 legacy=0x7f47bc071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47ac0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ac003f60 0x7f47ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47ac004d40 0x7f47ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc0c5880 con 0x7f47ac004d40 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47ac00bb30 con 0x7f47ac003f60 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f47ac00bd30 con 0x7f47bc154b90 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47ac0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c115e640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47ac004d40 0x7f47ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47bbfff640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ac003f60 0x7f47ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 -- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ac003f60 msgr2=0x7f47ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47ac003f60 0x7f47ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 -- 172.21.15.49:0/3630229074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47ac004d40 msgr2=0x7f47ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47ac004d40 0x7f47ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c095d640 1 -- 172.21.15.49:0/3630229074 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ac003520 con 0x7f47bc154b90 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.513+0000 7f47c115e640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47ac004d40 0x7f47ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.514+0000 7f47c095d640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47ac0039e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009ac0 tx=0x7f47ac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebeb2589867a4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.514+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0036070 con 0x7f47bc154b90 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b0031400 con 0x7f47bc154b90 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47bb7fe640 1 -- 172.21.15.49:0/3630229074 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0009690 con 0x7f47bc154b90 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 -- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 msgr2=0x7f47ac0039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/3630229074 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc154b90 0x7f47ac0039e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009ac0 tx=0x7f47ac005830 comp rx=0 tx=0).stop 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c115e640 1 -- 172.21.15.49:0/3630229074 reap_dead start 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 -- 172.21.15.49:0/3630229074 shutdown_connections 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 -- 172.21.15.49:0/3630229074 >> 172.21.15.49:0/3630229074 conn(0x7f47bc074840 msgr2=0x7f47bc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 -- 172.21.15.49:0/3630229074 shutdown_connections 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.515+0000 7f47c2bc1640 1 -- 172.21.15.49:0/3630229074 wait complete. 2024-05-09T08:58:01.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.516+0000 7f47c2bc1640 1 Processor -- start 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.516+0000 7f47c2bc1640 1 -- start start 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47bc154b90 0x7f47bc150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47bc1523b0 0x7f47bc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc0b4e50 0x7f47bc149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc15af40 con 0x7f47bc154b90 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc0c6120 con 0x7f47bc1523b0 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.517+0000 7f47c2bc1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc071f30 con 0x7f47bc0b4e50 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47bc154b90 0x7f47bc150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47bc154b90 0x7f47bc150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53892/0 (socket says 172.21.15.49:53892) 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 -- 172.21.15.49:0/2172957036 learned_addr learned my addr 172.21.15.49:0/2172957036 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47bc1523b0 0x7f47bc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc0b4e50 0x7f47bc149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 -- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc0b4e50 msgr2=0x7f47bc149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47bc0b4e50 0x7f47bc149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 -- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47bc1523b0 msgr2=0x7f47bc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47bc1523b0 0x7f47bc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc152ff0 con 0x7f47bc154b90 2024-05-09T08:58:01.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47bc154b90 0x7f47bc150aa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47b0034040 tx=0x7f47b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0040020 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b0003a40 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.518+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc14f9e0 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.519+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc0b6400 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.519+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0036040 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.520+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f47bc04f660 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.520+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47b0043020 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47b97fa640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47a006be30 0x7f47a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5886+0+0 (secure 0 0 0) 0x7f47b00aadd0 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v58) ==== 54+0+16 (secure 0 0 0) 0x7f47b00ac020 con 0x7f47bc154b90 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47a006be30 0x7f47a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.527 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f47bc0b45b0 0x7f47bc149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47bc0cd530 con 0x7f47bc0b45b0 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 180, 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-05-09T08:58:01.528 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:01.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.521+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f47bc0b45b0 0x7f47bc149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.522+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47a006be30 0x7f47a006e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f47bc15a280 tx=0x7f47a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.522+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f47bc0b45b0 0x7f47bc149990 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.522+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc0cd530 con 0x7f47bc0b45b0 2024-05-09T08:58:01.530 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-09T08:58:01.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.539+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc0b4990 con 0x7f47bc0b45b0 2024-05-09T08:58:01.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.540+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc0b4990 con 0x7f47bc0b45b0 2024-05-09T08:58:01.540 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.540+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f47bc1f38a0 0x7f47bc1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.540+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f47bc152070 con 0x7f47bc1f38a0 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.541+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f47bc1f38a0 0x7f47bc1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.541+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7f47bc1f38a0 0x7f47bc1f3c80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.541 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.542+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc152070 con 0x7f47bc1f38a0 2024-05-09T08:58:01.549 INFO:tasks.workunit:Stopping ['fs/misc/trivial_sync.sh'] on client.0... 2024-05-09T08:58:01.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T08:58:01.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.559+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc1f4410 con 0x7f47bc1f38a0 2024-05-09T08:58:01.560 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.561+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc1f4410 con 0x7f47bc1f38a0 2024-05-09T08:58:01.561 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.561+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=59}) -- 0x7f47bc0ce390 con 0x7f47bc154b90 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.561+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47bc0ce730 con 0x7f47bc154b90 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.561+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f47b007d410 con 0x7f47bc154b90 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.561+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f47bc061c70 0x7f47bc0620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.562+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f47bc0cf240 con 0x7f47bc061c70 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.562+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f47bc061c70 0x7f47bc0620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.562+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f47bc061c70 0x7f47bc0620c0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.563+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 6: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc0cf240 con 0x7f47bc061c70 2024-05-09T08:58:01.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.577+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc062850 con 0x7f47bc061c70 2024-05-09T08:58:01.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.580+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 7: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc062850 con 0x7f47bc061c70 2024-05-09T08:58:01.579 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-05-09T08:58:01.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-09T08:58:01.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.580+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f47bc02c550 0x7f47bc02c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.580+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 8: {"prefix": "get_command_descriptions"}) -- 0x7f47bc062c10 con 0x7f47bc02c550 2024-05-09T08:58:01.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.581+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f47bc02c550 0x7f47bc02c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.581+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f47bc02c550 0x7f47bc02c990 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.582+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 8: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc062c10 con 0x7f47bc02c550 2024-05-09T08:58:01.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.597+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 9: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc02d1a0 con 0x7f47bc02c550 2024-05-09T08:58:01.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.599+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 9: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc02d1a0 con 0x7f47bc02c550 2024-05-09T08:58:01.598 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-05-09T08:58:01.599 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-09T08:58:01.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.599+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f47bc203120 0x7f47bc205510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.599+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 10: {"prefix": "get_command_descriptions"}) -- 0x7f47bc062c10 con 0x7f47bc203120 2024-05-09T08:58:01.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.599+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f47bc203120 0x7f47bc205510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.600+0000 7f47c095d640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f47bc203120 0x7f47bc205510 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.601+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 10: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc062c10 con 0x7f47bc203120 2024-05-09T08:58:01.610 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:58:01.611+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:58:01.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.615+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 11: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc062c10 con 0x7f47bc203120 2024-05-09T08:58:01.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.617+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 11: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc062c10 con 0x7f47bc203120 2024-05-09T08:58:01.617 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-05-09T08:58:01.617 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-09T08:58:01.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.618+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f47bc205f60 0x7f47bc208350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.618+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f47bc062c10 con 0x7f47bc205f60 2024-05-09T08:58:01.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.618+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f47bc205f60 0x7f47bc208350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.618+0000 7f47bbfff640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f47bc205f60 0x7f47bc208350 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.619+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc062c10 con 0x7f47bc205f60 2024-05-09T08:58:01.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.634+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc062c10 con 0x7f47bc205f60 2024-05-09T08:58:01.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.636+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc062c10 con 0x7f47bc205f60 2024-05-09T08:58:01.635 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.636+0000 7f47c2bc1640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f47bc208da0 0x7f47bc20b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.636+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f47bc20df60 con 0x7f47bc208da0 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.636+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f47bc208da0 0x7f47bc20b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.637+0000 7f47c115e640 1 --2- 172.21.15.49:0/2172957036 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f47bc208da0 0x7f47bc20b190 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:01.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.637+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47bc20df60 con 0x7f47bc208da0 2024-05-09T08:58:01.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.652+0000 7f47c2bc1640 1 -- 172.21.15.49:0/2172957036 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f47bc20b6d0 con 0x7f47bc208da0 2024-05-09T08:58:01.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.654+0000 7f47b97fa640 1 -- 172.21.15.49:0/2172957036 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f47bc20b6d0 con 0x7f47bc208da0 2024-05-09T08:58:01.653 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-05-09T08:58:01.654 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-09T08:58:01.654 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:01.654 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:01.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-09T08:58:01.658 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-09T08:58:01.664 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-09T08:58:01.711 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:58:01.712+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:58:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.737+0000 7f4425ffb640 1 -- 172.21.15.49:0/2550029301 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["0"]}]=0 marked out osd.0. v59) ==== 87+0+0 (secure 0 0 0) 0x7f442007a2e0 con 0x7f4430154b90 2024-05-09T08:58:01.737 INFO:teuthology.orchestra.run.smithi049.stderr:marked out osd.0. 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 msgr2=0x7f441406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 0x7f441406e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f442c000f60 tx=0x7f442c009040 comp rx=0 tx=0).stop 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 msgr2=0x7f443014ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4420000c00 tx=0x7f44200046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 shutdown_connections 2024-05-09T08:58:01.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f441406be30 0x7f441406e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4430154f70 0x7f443014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4430154b90 0x7f443014ca10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 --2- 172.21.15.49:0/2550029301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f44300c6b80 0x7f44300b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 >> 172.21.15.49:0/2550029301 conn(0x7f4430074840 msgr2=0x7f4430159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.743+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 shutdown_connections 2024-05-09T08:58:01.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:01.744+0000 7f443799e640 1 -- 172.21.15.49:0/2550029301 wait complete. 2024-05-09T08:58:01.785 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:01.785 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:01.795 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-09T08:58:01.839 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:58:01.840+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:58:01.874 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:01.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:58:01.897 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:01.898 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:01.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-09T08:58:01.948 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-09T08:58:01.949+0000 7f6379bd3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36589) UID: 0 2024-05-09T08:58:02.013 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-09T08:58:02.014+0000 7f0bc4c92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36592) UID: 0 2024-05-09T08:58:02.051 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.051 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.060 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-09T08:58:02.114 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:58:02.115+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:58:02.202 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.203 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-09T08:58:02.255 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-09T08:58:02.256+0000 7f79af775640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36593) UID: 0 2024-05-09T08:58:02.313 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.313 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.315 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-05-09T08:58:02.316+0000 7f5de954d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 35642) UID: 0 2024-05-09T08:58:02.322 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-09T08:58:02.394 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T08:58:02.394 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-09T08:58:02.403 INFO:tasks.thrashosds:joining thrashosds 2024-05-09T08:58:02.464 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.465 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.473 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-09T08:58:02.576 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.576 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-09T08:58:02.686 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.686 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.695 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-09T08:58:02.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-09T08:58:02.809 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.809 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.818 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-09T08:58:02.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.871+0000 7fcf7f7a9640 1 Processor -- start 2024-05-09T08:58:02.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.871+0000 7fcf7f7a9640 1 -- start start 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7f7a9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154b90 0x7fcf780cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7f7a9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf78154f70 0x7fcf780cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7f7a9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf78072600 con 0x7fcf780c6640 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7f7a9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fcf780728e0 con 0x7fcf78154f70 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7f7a9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf780c5880 con 0x7fcf78154b90 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fcf78154f70 0x7fcf780cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7dd46640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fcf780c6640 0x7fcf780c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46552/0 (socket says 172.21.15.49:46552) 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7dd46640 1 -- 172.21.15.49:0/3608187494 learned_addr learned my addr 172.21.15.49:0/3608187494 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/3608187494 >> v2:172.21.15.49:3300/0 conn(0x7fcf78154f70 0x7fcf780cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35972/0 (socket says 172.21.15.49:35972) 2024-05-09T08:58:02.872 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7d545640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154b90 0x7fcf780cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 -- 172.21.15.49:0/3608187494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154b90 msgr2=0x7fcf780cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154b90 0x7fcf780cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 -- 172.21.15.49:0/3608187494 >> v1:172.21.15.49:6789/0 conn(0x7fcf780c6640 legacy=0x7fcf780c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:02.873 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.872+0000 7fcf7cd44640 1 -- 172.21.15.49:0/3608187494 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf780c5b60 con 0x7fcf78154f70 2024-05-09T08:58:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.873+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf78154f70 0x7fcf780cb7c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcf680097a0 tx=0x7fcf6802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305287c633564619 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.873+0000 7fcf6ffff640 1 -- 172.21.15.49:0/3608187494 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68037070 con 0x7fcf78154f70 2024-05-09T08:58:02.874 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.873+0000 7fcf6ffff640 1 -- 172.21.15.49:0/3608187494 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf68032440 con 0x7fcf78154f70 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.873+0000 7fcf6ffff640 1 -- 172.21.15.49:0/3608187494 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68032810 con 0x7fcf78154f70 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/3608187494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf78154f70 msgr2=0x7fcf780cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf78154f70 0x7fcf780cb7c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcf680097a0 tx=0x7fcf6802a210 comp rx=0 tx=0).stop 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/3608187494 shutdown_connections 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf78154f70 0x7fcf780cb7c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/3608187494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154b90 0x7fcf780cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/3608187494 >> 172.21.15.49:0/3608187494 conn(0x7fcf78074840 msgr2=0x7fcf78074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/3608187494 shutdown_connections 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.874+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/3608187494 wait complete. 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.875+0000 7fcf7f7a9640 1 Processor -- start 2024-05-09T08:58:02.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.875+0000 7fcf7f7a9640 1 -- start start 2024-05-09T08:58:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf78154b90 0x7fcf781e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154f70 0x7fcf781e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7815af40 con 0x7fcf780c6640 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf780c6120 con 0x7fcf78154b90 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7f7a9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fcf78071820 con 0x7fcf78154f70 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35980/0 (socket says 172.21.15.49:35980) 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 -- 172.21.15.49:0/1385591228 learned_addr learned my addr 172.21.15.49:0/1385591228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7dd46640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154f70 0x7fcf781e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf78154b90 0x7fcf781e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 -- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154f70 msgr2=0x7fcf781e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154f70 0x7fcf781e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 -- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf78154b90 msgr2=0x7fcf781e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:02.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf78154b90 0x7fcf781e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 -- 172.21.15.49:0/1385591228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781e10f0 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.876+0000 7fcf7d545640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e9e0 tx=0x7fcf7400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.877+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017070 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.877+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7400c9d0 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.877+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781f19e0 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.877+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400cdc0 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.877+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf78159c70 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.878+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78151f70 con 0x7fcf780c6640 2024-05-09T08:58:02.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.878+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf7401a070 con 0x7fcf780c6640 2024-05-09T08:58:02.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.879+0000 7fcf6dffb640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:02.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.879+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5998+0+0 (secure 0 0 0) 0x7fcf7408af50 con 0x7fcf780c6640 2024-05-09T08:58:02.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.879+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:02.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.880+0000 7fcf7cd44640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcf68034000 tx=0x7fcf6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:02.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:02.883+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf74059100 con 0x7fcf780c6640 2024-05-09T08:58:02.921 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:02.922 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:02.930 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-09T08:58:03.032 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:03.032 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:03.040 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-09T08:58:03.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.076+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fcf7804f660 con 0x7fcf780c6640 2024-05-09T08:58:03.142 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-09T08:58:03.142 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-09T08:58:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.746+0000 7fcf6dffb640 1 -- 172.21.15.49:0/1385591228 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v61) ==== 91+0+0 (secure 0 0 0) 0x7fcf7405d400 con 0x7fcf780c6640 2024-05-09T08:58:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-05-09T08:58:03.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 msgr2=0x7fcf5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.749 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcf68034000 tx=0x7fcf6802e000 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 msgr2=0x7fcf78159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e9e0 tx=0x7fcf7400eea0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 shutdown_connections 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fcf5c06be30 0x7fcf5c06e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.750+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf78154f70 0x7fcf781e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.751+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf78154b90 0x7fcf781e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.751+0000 7fcf7f7a9640 1 --2- 172.21.15.49:0/1385591228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fcf780c6640 0x7fcf78159240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.750 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.751+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 >> 172.21.15.49:0/1385591228 conn(0x7fcf78074840 msgr2=0x7fcf781538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:03.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.751+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 shutdown_connections 2024-05-09T08:58:03.751 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.752+0000 7fcf7f7a9640 1 -- 172.21.15.49:0/1385591228 wait complete. 2024-05-09T08:58:03.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-09T08:58:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.876+0000 7f689b431640 1 Processor -- start 2024-05-09T08:58:03.875 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.876+0000 7f689b431640 1 -- start start 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f689b431640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f689406bf80 0x7f689406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f689b431640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6894158000 con 0x7f689415ac60 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f689b431640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6894157480 con 0x7f68941512d0 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f689b431640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f689406c900 con 0x7f689406bf80 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68999ce640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f689415ac60 0x7f689415b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46560/0 (socket says 172.21.15.49:46560) 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68999ce640 1 -- 172.21.15.49:0/2665417294 learned_addr learned my addr 172.21.15.49:0/2665417294 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68989cc640 1 --2- 172.21.15.49:0/2665417294 >> v2:172.21.15.183:3300/0 conn(0x7f689406bf80 0x7f689406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040064334 0 0) 0x7f6894158000 con 0x7f689415ac60 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6880003520 con 0x7f689415ac60 2024-05-09T08:58:03.877 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68989cc640 1 -- 172.21.15.49:0/2665417294 >> v1:172.21.15.183:6789/0 conn(0x7f68941512d0 legacy=0x7f689406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231530975 0 0) 0x7f6894157480 con 0x7f68941512d0 2024-05-09T08:58:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68989cc640 1 -- 172.21.15.49:0/2665417294 >> v1:172.21.15.49:6789/0 conn(0x7f689415ac60 legacy=0x7f689415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.878 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.877+0000 7f68989cc640 1 -- 172.21.15.49:0/2665417294 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6894157760 con 0x7f689406bf80 2024-05-09T08:58:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.878+0000 7f68989cc640 1 --2- 172.21.15.49:0/2665417294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f689406bf80 0x7f689406c3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f688402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3fe310b1a9c3ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.878+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884037070 con 0x7f689406bf80 2024-05-09T08:58:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.878+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6884032640 con 0x7f689406bf80 2024-05-09T08:58:03.879 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.878+0000 7f688bfff640 1 -- 172.21.15.49:0/2665417294 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884032a10 con 0x7f689406bf80 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 -- 172.21.15.49:0/2665417294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f689406bf80 msgr2=0x7f689406c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 --2- 172.21.15.49:0/2665417294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f689406bf80 0x7f689406c3c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f68840097a0 tx=0x7f688402a2c0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 -- 172.21.15.49:0/2665417294 shutdown_connections 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 --2- 172.21.15.49:0/2665417294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f689406bf80 0x7f689406c3c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 -- 172.21.15.49:0/2665417294 >> 172.21.15.49:0/2665417294 conn(0x7f68940cc290 msgr2=0x7f68940cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 -- 172.21.15.49:0/2665417294 shutdown_connections 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.879+0000 7f689b431640 1 -- 172.21.15.49:0/2665417294 wait complete. 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.880+0000 7f689b431640 1 Processor -- start 2024-05-09T08:58:03.880 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.880+0000 7f689b431640 1 -- start start 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f689406bf80 0x7f6894153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 0x7f68940cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68940c8af0 con 0x7f689415ac60 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6894157d20 con 0x7f68941512d0 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f689b431640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f68941570b0 con 0x7f689406bf80 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35984/0 (socket says 172.21.15.49:35984) 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 -- 172.21.15.49:0/1992534869 learned_addr learned my addr 172.21.15.49:0/1992534869 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68991cd640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f689406bf80 0x7f6894153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:03.883 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68989cc640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 0x7f68940cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 -- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f689406bf80 msgr2=0x7f6894153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f689406bf80 0x7f6894153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 -- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 msgr2=0x7f68940cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 0x7f68940cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68999ce640 1 -- 172.21.15.49:0/1992534869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68940ca330 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.881+0000 7f68989cc640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 0x7f68940cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f68999ce640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f688c00cad0 tx=0x7f688c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c017070 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c00ac30 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6894158690 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c012400 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.882+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68940ca5d0 con 0x7f689415ac60 2024-05-09T08:58:03.884 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.883+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f689404f660 con 0x7f689415ac60 2024-05-09T08:58:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.884+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f688c01a070 con 0x7f689415ac60 2024-05-09T08:58:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.884+0000 7f6889ffb640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 0x7f687806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:03.885 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.884+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5998+0+0 (secure 0 0 0) 0x7f688c08ad60 con 0x7f689415ac60 2024-05-09T08:58:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.884+0000 7f68991cd640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 0x7f687806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:03.886 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.885+0000 7f68991cd640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 0x7f687806e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:03.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:03.888+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f688c0595f0 con 0x7f689415ac60 2024-05-09T08:58:04.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.084+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f689406cac0 con 0x7f689415ac60 2024-05-09T08:58:04.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.251+0000 7f6889ffb640 1 -- 172.21.15.49:0/1992534869 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v62) ==== 101+0+0 (secure 0 0 0) 0x7f688c058ef0 con 0x7f689415ac60 2024-05-09T08:58:04.250 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 msgr2=0x7f687806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 0x7f687806e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f60 tx=0x7f6890009040 comp rx=0 tx=0).stop 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 msgr2=0x7f6894151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f688c00cad0 tx=0x7f688c00cf90 comp rx=0 tx=0).stop 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 shutdown_connections 2024-05-09T08:58:04.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f687806be30 0x7f687806e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f689415ac60 0x7f6894151a70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68941512d0 0x7f68940cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 --2- 172.21.15.49:0/1992534869 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f689406bf80 0x7f6894153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 >> 172.21.15.49:0/1992534869 conn(0x7f68940cc290 msgr2=0x7f6894155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 shutdown_connections 2024-05-09T08:58:04.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:04.254+0000 7f689b431640 1 -- 172.21.15.49:0/1992534869 wait complete. 2024-05-09T08:58:06.759 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-09T08:58:06.759 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-05-09T08:58:06.759 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T08:58:06.760 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T08:58:06.762 INFO:tasks.ceph.osd.2:Started 2024-05-09T08:58:06.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:58:06.878 INFO:teuthology.orchestra.run.smithi049.stderr:no valid command found; 10 closest matches: 2024-05-09T08:58:06.878 INFO:teuthology.orchestra.run.smithi049.stderr:0 2024-05-09T08:58:06.878 INFO:teuthology.orchestra.run.smithi049.stderr:1 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:2 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:abort 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:assert 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:config diff 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:config diff get 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:config get 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:config help [] 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:config set ... 2024-05-09T08:58:06.879 INFO:teuthology.orchestra.run.smithi049.stderr:admin_socket: invalid command 2024-05-09T08:58:06.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-09T08:58:06.881 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-05-09T08:58:07.368 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:58:07.367+0000 7f3444610640 -1 Falling back to public interface 2024-05-09T08:58:10.837 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:58:10.838+0000 7f3444610640 -1 osd.2 38 log_to_monitors true 2024-05-09T08:58:10.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:58:10.878 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:58:11.296 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-05-09T08:58:11.296+0000 7f3437cac640 -1 osd.2 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T08:58:11.882 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-09T08:58:12.007 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-05-09T08:58:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-05-09T08:58:12.008 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:12.017 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-09T08:58:12.017 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-05-09T08:58:12.017 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-05-09T08:58:12.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.123+0000 7f4f4d0a5640 1 Processor -- start 2024-05-09T08:58:12.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 -- start start 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f480c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4f4815af60 0x7f4f480c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f480c4b10 con 0x7f4f480c71d0 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f4f480c5c70 con 0x7f4f4815af60 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.124+0000 7f4f4d0a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f480c4da0 con 0x7f4f48151be0 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f4759e640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4f480c71d0 0x7f4f4806dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:46574/0 (socket says 172.21.15.49:46574) 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f4759e640 1 -- 172.21.15.49:0/2072106221 learned_addr learned my addr 172.21.15.49:0/2072106221 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:12.127 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2072106221 >> v2:172.21.15.183:3300/0 conn(0x7f4f4815af60 0x7f4f480c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f480c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904965545 0 0) 0x7f4f480c4b10 con 0x7f4f480c71d0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f34003520 con 0x7f4f480c71d0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 71609659 0 0) 0x7f4f34003520 con 0x7f4f480c71d0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 msgr2=0x7f4f480c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f480c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 >> v2:172.21.15.183:3300/0 conn(0x7f4f4815af60 msgr2=0x7f4f480c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 --2- 172.21.15.49:0/2072106221 >> v2:172.21.15.183:3300/0 conn(0x7f4f4815af60 0x7f4f480c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f480c5060 con 0x7f4f480c71d0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f4f400019f0 con 0x7f4f480c71d0 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 >> v1:172.21.15.49:6789/0 conn(0x7f4f480c71d0 legacy=0x7f4f4806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:12.128 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480c5c70 con 0x7f4f340039b0 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480c4da0 con 0x7f4f34004800 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3400b9f0 con 0x7f4f48151be0 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.125+0000 7f4f4759e640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 -- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 msgr2=0x7f4f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f48151be0 0x7f4f340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 -- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34004800 msgr2=0x7f4f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.129 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 -- 172.21.15.49:0/2072106221 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f34003520 con 0x7f4f340039b0 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485c2fcb971c4425 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38036070 con 0x7f4f340039b0 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.126+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f38031400 con 0x7f4f340039b0 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f45d9b640 1 -- 172.21.15.49:0/2072106221 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38009690 con 0x7f4f340039b0 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2072106221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f340039b0 msgr2=0x7f4f34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2072106221 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).stop 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4759e640 1 -- 172.21.15.49:0/2072106221 reap_dead start 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2072106221 shutdown_connections 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2072106221 >> 172.21.15.49:0/2072106221 conn(0x7f4f480cc290 msgr2=0x7f4f480cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2072106221 shutdown_connections 2024-05-09T08:58:12.130 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.127+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2072106221 wait complete. 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 Processor -- start 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 -- start start 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f340039b0 0x7f4f4815a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 0x7f4f481ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480c5900 con 0x7f4f48149680 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f480c5620 con 0x7f4f48154520 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4d0a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f48151fc0 con 0x7f4f340039b0 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:36012/0 (socket says 172.21.15.49:36012) 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.129+0000 7f4f4659c640 1 -- 172.21.15.49:0/2544734850 learned_addr learned my addr 172.21.15.49:0/2544734850 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f340039b0 0x7f4f4815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4759e640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 0x7f4f481ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 -- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f340039b0 msgr2=0x7f4f4815a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f340039b0 0x7f4f4815a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 -- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 msgr2=0x7f4f481ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 0x7f4f481ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 -- 172.21.15.49:0/2544734850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4814a080 con 0x7f4f48149680 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4759e640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 0x7f4f481ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4659c640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4f300116d0 tx=0x7f4f30011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000f9c0 con 0x7f4f48149680 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3000fb60 con 0x7f4f48149680 2024-05-09T08:58:12.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30015460 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4814a320 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.130+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481f2250 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.131+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4804f660 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.132+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f30015600 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.132+0000 7f4f3f7fe640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 0x7f4f2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.132+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5973+0+0 (secure 0 0 0) 0x7f4f3008b760 con 0x7f4f48149680 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.133+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 0x7f4f2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.133+0000 7f4f46d9d640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 0x7f4f2806e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f38033000 tx=0x7f4f38008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:12.136 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.137+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f30019030 con 0x7f4f48149680 2024-05-09T08:58:12.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:12.340+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd in", "ids": ["0"]} v 0) -- 0x7f4f4814bc60 con 0x7f4f48149680 2024-05-09T08:58:12.339 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-05-09T08:58:12.340+0000 7fbf24104640 -1 mon.a@0(leader).osd e65 definitely_dead 0 2024-05-09T08:58:13.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.273+0000 7f4f3f7fe640 1 -- 172.21.15.49:0/2544734850 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["0"]}]=0 marked in osd.0. v66) ==== 85+0+0 (secure 0 0 0) 0x7f4f3005a090 con 0x7f4f48149680 2024-05-09T08:58:13.273 INFO:teuthology.orchestra.run.smithi049.stderr:marked in osd.0. 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 msgr2=0x7f4f2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 0x7f4f2806e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f38033000 tx=0x7f4f38008030 comp rx=0 tx=0).stop 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 msgr2=0x7f4f48153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4f300116d0 tx=0x7f4f30011b90 comp rx=0 tx=0).stop 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 shutdown_connections 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f2806be60 0x7f4f2806e320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f48154520 0x7f4f481ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f48149680 0x7f4f48153fe0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4f300116d0 tx=0x7f4f30011b90 comp rx=0 tx=0).stop 2024-05-09T08:58:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 --2- 172.21.15.49:0/2544734850 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f340039b0 0x7f4f4815a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.278+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 >> 172.21.15.49:0/2544734850 conn(0x7f4f480cc290 msgr2=0x7f4f48150180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:13.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.279+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 shutdown_connections 2024-05-09T08:58:13.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.280+0000 7f4f4d0a5640 1 -- 172.21.15.49:0/2544734850 wait complete. 2024-05-09T08:58:13.303 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-05-09T08:58:13.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-09T08:58:13.408 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.409+0000 7f95caab3640 1 Processor -- start 2024-05-09T08:58:13.409 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.409+0000 7f95caab3640 1 -- start start 2024-05-09T08:58:13.418 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95caab3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95c41512d0 0x7f95c406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95caab3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 0x7f95c406bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95caab3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c4158000 con 0x7f95c406c4c0 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95caab3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f95c406c8e0 con 0x7f95c41512d0 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95caab3640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4157480 con 0x7f95c406b9f0 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95c41512d0 0x7f95c406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95c41512d0 0x7f95c406b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48176/0 (socket says 172.21.15.49:48176) 2024-05-09T08:58:13.419 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 -- 172.21.15.49:0/2618041099 learned_addr learned my addr 172.21.15.49:0/2618041099 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c3fff640 1 --2- 172.21.15.49:0/2618041099 >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 0x7f95c406bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 -- 172.21.15.49:0/2618041099 >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 msgr2=0x7f95c406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 --2- 172.21.15.49:0/2618041099 >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 0x7f95c406bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 -- 172.21.15.49:0/2618041099 >> v1:172.21.15.183:6789/0 conn(0x7f95c406c4c0 legacy=0x7f95c415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c884f640 1 -- 172.21.15.49:0/2618041099 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c4157760 con 0x7f95c41512d0 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.410+0000 7f95c3fff640 1 --2- 172.21.15.49:0/2618041099 >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 0x7f95c406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.411+0000 7f95c884f640 1 --2- 172.21.15.49:0/2618041099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c41512d0 0x7f95c406b4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95c40701d0 tx=0x7f95b802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a34d12eba50f820 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.420 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.411+0000 7f95c37fe640 1 -- 172.21.15.49:0/2618041099 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037070 con 0x7f95c41512d0 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.411+0000 7f95c37fe640 1 -- 172.21.15.49:0/2618041099 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8032440 con 0x7f95c41512d0 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.411+0000 7f95c37fe640 1 -- 172.21.15.49:0/2618041099 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8032810 con 0x7f95c41512d0 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 -- 172.21.15.49:0/2618041099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c41512d0 msgr2=0x7f95c406b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 --2- 172.21.15.49:0/2618041099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c41512d0 0x7f95c406b4b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95c40701d0 tx=0x7f95b802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 -- 172.21.15.49:0/2618041099 shutdown_connections 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 --2- 172.21.15.49:0/2618041099 >> v2:172.21.15.183:3300/0 conn(0x7f95c406b9f0 0x7f95c406bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 --2- 172.21.15.49:0/2618041099 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c41512d0 0x7f95c406b4b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.421 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 -- 172.21.15.49:0/2618041099 >> 172.21.15.49:0/2618041099 conn(0x7f95c40cc290 msgr2=0x7f95c40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 -- 172.21.15.49:0/2618041099 shutdown_connections 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.412+0000 7f95caab3640 1 -- 172.21.15.49:0/2618041099 wait complete. 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.413+0000 7f95caab3640 1 Processor -- start 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.413+0000 7f95caab3640 1 -- start start 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 0x7f95c40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c406c4c0 0x7f95c4151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41512d0 0x7f95c4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c8af0 con 0x7f95c406b9f0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4157d20 con 0x7f95c406c4c0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95caab3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95c41570b0 con 0x7f95c41512d0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 0x7f95c40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 0x7f95c40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48182/0 (socket says 172.21.15.49:48182) 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 -- 172.21.15.49:0/3331304144 learned_addr learned my addr 172.21.15.49:0/3331304144 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41512d0 0x7f95c4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c3fff640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c406c4c0 0x7f95c4151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41512d0 msgr2=0x7f95c4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41512d0 0x7f95c4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c406c4c0 msgr2=0x7f95c4151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c406c4c0 0x7f95c4151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c4149e40 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c884f640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 0x7f95c40cb5d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.414+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037040 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.415+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8042070 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.415+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8030bb0 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.415+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c414a120 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.415+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c414a6e0 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.415+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f95c404f660 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.416+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95b8047460 con 0x7f95c406b9f0 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f95a806be30 0x7f95a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.424 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5886+0+0 (secure 0 0 0) 0x7f95b80aba70 con 0x7f95c406b9f0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7f95b80b0300 con 0x7f95c406b9f0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c3fff640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f95a806be30 0x7f95a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 0x7f95c4143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c53c0 con 0x7f95c40c4ce0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 0x7f95c4143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c3fff640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f95a806be30 0x7f95a806e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95c40c93a0 tx=0x7f95b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.417+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f95c406cac0 con 0x7f95c40c4ce0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 0x7f95c4143ab0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc00fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc018030 con 0x7f95c40c4ce0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95bc006b50 con 0x7f95c40c4ce0 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 msgr2=0x7f95c4143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 0x7f95c4143ab0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc00fd70 comp rx=0 tx=0).stop 2024-05-09T08:58:13.425 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 msgr2=0x7f95c4143ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.418+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c40c4ce0 0x7f95c4143ab0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.424+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 0x7f95c41443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.424+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4144910 con 0x7f95c4143ff0 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.424+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 0x7f95c41443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.425+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c40c90f0 con 0x7f95c4143ff0 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.425+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 0x7f95c41443d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.425+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc02c0a0 con 0x7f95c4143ff0 2024-05-09T08:58:13.427 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.426+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.0 v2:172.21.15.49:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95bc007580 con 0x7f95c4143ff0 2024-05-09T08:58:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.426+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 msgr2=0x7f95c41443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.426+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 0x7f95c41443d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc003ae0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.426+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 msgr2=0x7f95c41443d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.426+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c4143ff0 0x7f95c41443d0 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc003ae0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.428 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a: {} 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.427+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 0x7f95c4142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.427+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4073cf0 con 0x7f95c4142850 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.427+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 0x7f95c4142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.427+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f95c4149020 con 0x7f95c4142850 2024-05-09T08:58:13.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.427+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 0x7f95c4142c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95bc01e450 tx=0x7f95bc00fd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc014da0 con 0x7f95c4142850 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.1 v2:172.21.15.183:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95bc014da0 con 0x7f95c4142850 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 msgr2=0x7f95c4142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 0x7f95c4142c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95bc01e450 tx=0x7f95bc00fd70 comp rx=0 tx=0).stop 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 msgr2=0x7f95c4142c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c4142850 0x7f95c4142c60 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95bc01e450 tx=0x7f95bc00fd70 comp rx=0 tx=0).stop 2024-05-09T08:58:13.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.428+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 reap_dead start 2024-05-09T08:58:13.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.435+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 0x7f95c4151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.435+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40cbb10 con 0x7f95c41431a0 2024-05-09T08:58:13.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.435+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 0x7f95c4151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.435+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c4143580 con 0x7f95c41431a0 2024-05-09T08:58:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.435+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 0x7f95c4151d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000f80 tx=0x7f95bc02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.436+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc03c9f0 con 0x7f95c41431a0 2024-05-09T08:58:13.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.1 v2:172.21.15.183:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95bc034670 con 0x7f95c41431a0 2024-05-09T08:58:13.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 msgr2=0x7f95c4151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 0x7f95c4151d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000f80 tx=0x7f95bc02a000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 msgr2=0x7f95c4151d40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95c41431a0 0x7f95c4151d40 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95bc000f80 tx=0x7f95bc02a000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.437 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b: {} 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 0x7f95c4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.437+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40c60b0 con 0x7f95c41518b0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.438+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 0x7f95c4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.438+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f95c40c6260 con 0x7f95c41518b0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.438+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 0x7f95c4152280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.438+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc034c30 con 0x7f95c41518b0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.439+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.2 v2:172.21.15.183:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f95bc001020 con 0x7f95c41518b0 2024-05-09T08:58:13.438 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.439+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 msgr2=0x7f95c4152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.439+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 0x7f95c4152280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc024000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.439+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 msgr2=0x7f95c4152280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.439 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.439+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c41518b0 0x7f95c4152280 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00f8b0 tx=0x7f95bc024000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.445+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 0x7f95c4155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.445+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40c3e70 con 0x7f95c4155470 2024-05-09T08:58:13.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.445+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 0x7f95c4155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.445+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f95c4155d90 con 0x7f95c4155470 2024-05-09T08:58:13.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.446+0000 7f95c9050640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 0x7f95c4155850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d5c0 tx=0x7f95bc035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.446+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc03ce60 con 0x7f95c4155470 2024-05-09T08:58:13.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.447+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 <== mon.2 v2:172.21.15.183:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f95bc053050 con 0x7f95c4155470 2024-05-09T08:58:13.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.447+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 msgr2=0x7f95c4155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.447+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 0x7f95c4155850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d5c0 tx=0x7f95bc035000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.447+0000 7f95c17fa640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 msgr2=0x7f95c4155850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-05-09T08:58:13.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.447+0000 7f95c17fa640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95c4155470 0x7f95c4155850 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00d5c0 tx=0x7f95bc035000 comp rx=0 tx=0).stop 2024-05-09T08:58:13.447 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c: {} 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f95a806be30 msgr2=0x7f95a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f95a806be30 0x7f95a806e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f95c40c93a0 tx=0x7f95b0008040 comp rx=0 tx=0).stop 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 msgr2=0x7f95c40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 --2- 172.21.15.49:0/3331304144 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95c406b9f0 0x7f95c40cb5d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f95b8000c00 tx=0x7f95b80046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95c9050640 1 -- 172.21.15.49:0/3331304144 reap_dead start 2024-05-09T08:58:13.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 shutdown_connections 2024-05-09T08:58:13.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 >> 172.21.15.49:0/3331304144 conn(0x7f95c40cc290 msgr2=0x7f95c40c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:13.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 shutdown_connections 2024-05-09T08:58:13.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.448+0000 7f95caab3640 1 -- 172.21.15.49:0/3331304144 wait complete. 2024-05-09T08:58:13.458 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-09T08:58:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.566+0000 7f7aa53f9640 1 Processor -- start 2024-05-09T08:58:13.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.566+0000 7f7aa53f9640 1 -- start start 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7aa53f9640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7aa0154df0 0x7f7aa015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7aa53f9640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7aa00c7e80 0x7f7aa006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7aa53f9640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa015c700 con 0x7f7aa01551d0 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7aa53f9640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa0071ef0 con 0x7f7aa0154df0 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7aa53f9640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa015ed50 con 0x7f7aa00c7e80 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9effd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7aa0154df0 0x7f7aa015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a97fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f7aa01551d0 0x7f7aa015e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51184/0 (socket says 172.21.15.49:51184) 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a97fff640 1 -- 172.21.15.49:0/2992129316 learned_addr learned my addr 172.21.15.49:0/2992129316 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9f7fe640 1 --2- 172.21.15.49:0/2992129316 >> v2:172.21.15.183:3300/0 conn(0x7f7aa00c7e80 0x7f7aa006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9effd640 1 -- 172.21.15.49:0/2992129316 >> v2:172.21.15.183:3300/0 conn(0x7f7aa00c7e80 msgr2=0x7f7aa006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9effd640 1 --2- 172.21.15.49:0/2992129316 >> v2:172.21.15.183:3300/0 conn(0x7f7aa00c7e80 0x7f7aa006f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9effd640 1 -- 172.21.15.49:0/2992129316 >> v1:172.21.15.49:6789/0 conn(0x7f7aa01551d0 legacy=0x7f7aa015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9e7fc640 1 -- 172.21.15.49:0/2992129316 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3851460252 0 0) 0x7f7aa015c700 con 0x7f7aa01551d0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.567+0000 7f7a9effd640 1 -- 172.21.15.49:0/2992129316 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa015f030 con 0x7f7aa0154df0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7a9effd640 1 --2- 172.21.15.49:0/2992129316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa015c1c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7aa006fd10 tx=0x7f7a9002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30633ee045ddc354 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7a9e7fc640 1 -- 172.21.15.49:0/2992129316 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90037070 con 0x7f7aa0154df0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7a9e7fc640 1 -- 172.21.15.49:0/2992129316 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a90032440 con 0x7f7aa0154df0 2024-05-09T08:58:13.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7a9e7fc640 1 -- 172.21.15.49:0/2992129316 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90032810 con 0x7f7aa0154df0 2024-05-09T08:58:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7aa53f9640 1 -- 172.21.15.49:0/2992129316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 msgr2=0x7f7aa015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.568+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/2992129316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa015c1c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7aa006fd10 tx=0x7f7a9002a230 comp rx=0 tx=0).stop 2024-05-09T08:58:13.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 -- 172.21.15.49:0/2992129316 shutdown_connections 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/2992129316 >> v2:172.21.15.183:3300/0 conn(0x7f7aa00c7e80 0x7f7aa006f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/2992129316 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa015c1c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 -- 172.21.15.49:0/2992129316 >> 172.21.15.49:0/2992129316 conn(0x7f7aa006b200 msgr2=0x7f7aa006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 -- 172.21.15.49:0/2992129316 shutdown_connections 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.569+0000 7f7aa53f9640 1 -- 172.21.15.49:0/2992129316 wait complete. 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.570+0000 7f7aa53f9640 1 Processor -- start 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.570+0000 7f7aa53f9640 1 -- start start 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa00c7e80 0x7f7aa00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7aa00ce100 0x7f7aa01f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa015b480 con 0x7f7aa0154df0 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa015b1a0 con 0x7f7aa00c7e80 2024-05-09T08:58:13.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa015e9d0 con 0x7f7aa00ce100 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48228/0 (socket says 172.21.15.49:48228) 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 -- 172.21.15.49:0/616134273 learned_addr learned my addr 172.21.15.49:0/616134273 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a9f7fe640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7aa00ce100 0x7f7aa01f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a9effd640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa00c7e80 0x7f7aa00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 -- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7aa00ce100 msgr2=0x7f7aa01f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7aa00ce100 0x7f7aa01f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 -- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa00c7e80 msgr2=0x7f7aa00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa00c7e80 0x7f7aa00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 -- 172.21.15.49:0/616134273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa00ccbb0 con 0x7f7aa0154df0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a97fff640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c410 tx=0x7f7a8800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880074e0 con 0x7f7aa0154df0 2024-05-09T08:58:13.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a880047e0 con 0x7f7aa0154df0 2024-05-09T08:58:13.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa00cbae0 con 0x7f7aa0154df0 2024-05-09T08:58:13.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88004bd0 con 0x7f7aa0154df0 2024-05-09T08:58:13.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.571+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa00cb520 con 0x7f7aa0154df0 2024-05-09T08:58:13.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.572+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa004f660 con 0x7f7aa0154df0 2024-05-09T08:58:13.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.573+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a88011070 con 0x7f7aa0154df0 2024-05-09T08:58:13.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.573+0000 7f7a977fe640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 0x7f7a8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:13.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.573+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5886+0+0 (secure 0 0 0) 0x7f7a8808a9c0 con 0x7f7aa0154df0 2024-05-09T08:58:13.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.574+0000 7f7a9effd640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 0x7f7a8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:13.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.574+0000 7f7a9effd640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 0x7f7a8006e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a90034000 tx=0x7f7a9002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:13.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.578+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a8808ce20 con 0x7f7aa0154df0 2024-05-09T08:58:13.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:13.771+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f7aa0152f80 con 0x7f7aa0154df0 2024-05-09T08:58:14.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.253+0000 7f7a977fe640 1 -- 172.21.15.49:0/616134273 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v67) ==== 113+0+0 (secure 0 0 0) 0x7f7a88058c80 con 0x7f7aa0154df0 2024-05-09T08:58:14.252 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-05-09T08:58:14.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.257+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 msgr2=0x7f7a8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.257+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 0x7f7a8006e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a90034000 tx=0x7f7a9002e000 comp rx=0 tx=0).stop 2024-05-09T08:58:14.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.257+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 msgr2=0x7f7aa01f4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.257+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c410 tx=0x7f7a8800c8d0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 shutdown_connections 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a8006be30 0x7f7a8006e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7aa00ce100 0x7f7aa01f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7aa0154df0 0x7f7aa01f4750 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 --2- 172.21.15.49:0/616134273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7aa00c7e80 0x7f7aa00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.258+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 >> 172.21.15.49:0/616134273 conn(0x7f7aa006b200 msgr2=0x7f7aa0071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:14.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.261+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 shutdown_connections 2024-05-09T08:58:14.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.261+0000 7f7aa53f9640 1 -- 172.21.15.49:0/616134273 wait complete. 2024-05-09T08:58:14.281 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-09T08:58:14.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.390+0000 7f15481ab640 1 Processor -- start 2024-05-09T08:58:14.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 -- start start 2024-05-09T08:58:14.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1540154b90 0x7f15400cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 0x7f15400cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1540072600 con 0x7f15400c6640 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f15400728e0 con 0x7f1540154b90 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.391+0000 7f15481ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f15400c5880 con 0x7f1540154f70 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1546748640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f15400c6640 0x7f15400c6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51190/0 (socket says 172.21.15.49:51190) 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1546748640 1 -- 172.21.15.49:0/1656268310 learned_addr learned my addr 172.21.15.49:0/1656268310 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545746640 1 --2- 172.21.15.49:0/1656268310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 0x7f15400cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545f47640 1 --2- 172.21.15.49:0/1656268310 >> v2:172.21.15.183:3300/0 conn(0x7f1540154b90 0x7f15400cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1544f45640 1 -- 172.21.15.49:0/1656268310 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 229776742 0 0) 0x7f1540072600 con 0x7f15400c6640 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1544f45640 1 -- 172.21.15.49:0/1656268310 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f152c003520 con 0x7f15400c6640 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545746640 1 -- 172.21.15.49:0/1656268310 >> v2:172.21.15.183:3300/0 conn(0x7f1540154b90 msgr2=0x7f15400cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545746640 1 --2- 172.21.15.49:0/1656268310 >> v2:172.21.15.183:3300/0 conn(0x7f1540154b90 0x7f15400cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545746640 1 -- 172.21.15.49:0/1656268310 >> v1:172.21.15.49:6789/0 conn(0x7f15400c6640 legacy=0x7f15400c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.392+0000 7f1545746640 1 -- 172.21.15.49:0/1656268310 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15400c5b60 con 0x7f1540154f70 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.393+0000 7f1545746640 1 --2- 172.21.15.49:0/1656268310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 0x7f15400cb7c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15300097a0 tx=0x7f153002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd1f486a4d3de5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.393+0000 7f1544f45640 1 -- 172.21.15.49:0/1656268310 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530037070 con 0x7f1540154f70 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.393+0000 7f1544f45640 1 -- 172.21.15.49:0/1656268310 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530032440 con 0x7f1540154f70 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.393+0000 7f1544f45640 1 -- 172.21.15.49:0/1656268310 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530032810 con 0x7f1540154f70 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 -- 172.21.15.49:0/1656268310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 msgr2=0x7f15400cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 --2- 172.21.15.49:0/1656268310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 0x7f15400cb7c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f15300097a0 tx=0x7f153002a210 comp rx=0 tx=0).stop 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 -- 172.21.15.49:0/1656268310 shutdown_connections 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 --2- 172.21.15.49:0/1656268310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1540154f70 0x7f15400cb7c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 --2- 172.21.15.49:0/1656268310 >> v2:172.21.15.183:3300/0 conn(0x7f1540154b90 0x7f15400cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 -- 172.21.15.49:0/1656268310 >> 172.21.15.49:0/1656268310 conn(0x7f1540074840 msgr2=0x7f1540074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 -- 172.21.15.49:0/1656268310 shutdown_connections 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.394+0000 7f15481ab640 1 -- 172.21.15.49:0/1656268310 wait complete. 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.395+0000 7f15481ab640 1 Processor -- start 2024-05-09T08:58:14.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.395+0000 7f15481ab640 1 -- start start 2024-05-09T08:58:14.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15400c6640 0x7f154014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1540154f70 0x7f1540150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f154015af40 con 0x7f1540154b90 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15400c6120 con 0x7f1540154f70 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1540071820 con 0x7f15400c6640 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48242/0 (socket says 172.21.15.49:48242) 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 -- 172.21.15.49:0/1470797259 learned_addr learned my addr 172.21.15.49:0/1470797259 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545f47640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15400c6640 0x7f154014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1546748640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1540154f70 0x7f1540150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 -- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15400c6640 msgr2=0x7f154014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15400c6640 0x7f154014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 -- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1540154f70 msgr2=0x7f1540150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1540154f70 0x7f1540150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 -- 172.21.15.49:0/1470797259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154014f520 con 0x7f1540154b90 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f1545746640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f15300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530041030 con 0x7f1540154b90 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153003d4e0 con 0x7f1540154b90 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530037040 con 0x7f1540154b90 2024-05-09T08:58:14.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15400ca610 con 0x7f1540154b90 2024-05-09T08:58:14.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.396+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1540151a80 con 0x7f1540154b90 2024-05-09T08:58:14.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.397+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15400d1180 con 0x7f1540154b90 2024-05-09T08:58:14.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.398+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1530044070 con 0x7f1540154b90 2024-05-09T08:58:14.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.398+0000 7f15367fc640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 0x7f152406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:14.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.399+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5911+0+0 (secure 0 0 0) 0x7f15300ab4d0 con 0x7f1540154b90 2024-05-09T08:58:14.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.399+0000 7f1545f47640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 0x7f152406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:14.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.399+0000 7f1545f47640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 0x7f152406e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153c009b40 tx=0x7f153c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:14.402 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.403+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f15400d1180 con 0x7f1540154b90 2024-05-09T08:58:14.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:14.597+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7f1540151eb0 con 0x7f1540154b90 2024-05-09T08:58:15.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.254+0000 7f15367fc640 1 -- 172.21.15.49:0/1470797259 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v68) ==== 133+0+0 (secure 0 0 0) 0x7f1530079770 con 0x7f1540154b90 2024-05-09T08:58:15.254 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-09T08:58:15.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 msgr2=0x7f152406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:15.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 0x7f152406e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f153c009b40 tx=0x7f153c009190 comp rx=0 tx=0).stop 2024-05-09T08:58:15.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 msgr2=0x7f1540150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f15300098c0 tx=0x7f15300046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 shutdown_connections 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f152406be30 0x7f152406e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1540154f70 0x7f1540150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1540154b90 0x7f1540150a10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 --2- 172.21.15.49:0/1470797259 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15400c6640 0x7f154014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 >> 172.21.15.49:0/1470797259 conn(0x7f1540074840 msgr2=0x7f15400b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:15.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 shutdown_connections 2024-05-09T08:58:15.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.258+0000 7f15481ab640 1 -- 172.21.15.49:0/1470797259 wait complete. 2024-05-09T08:58:15.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-09T08:58:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.383+0000 7f2645edd640 1 Processor -- start 2024-05-09T08:58:15.382 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.383+0000 7f2645edd640 1 -- start start 2024-05-09T08:58:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2640155180 0x7f26400c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f264006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f264006b740 0x7f26400c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f26400c7b90 con 0x7f264006b740 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f264015f160 con 0x7f2640155180 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f2645edd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f26400c78b0 con 0x7f26400c8f90 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f264006b740 0x7f26400c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f264006b740 0x7f26400c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48244/0 (socket says 172.21.15.49:48244) 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 -- 172.21.15.49:0/2657583022 learned_addr learned my addr 172.21.15.49:0/2657583022 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263effd640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f264006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263f7fe640 1 --2- 172.21.15.49:0/2657583022 >> v2:172.21.15.183:3300/0 conn(0x7f2640155180 0x7f26400c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 -- 172.21.15.49:0/2657583022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 msgr2=0x7f264006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f264006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 -- 172.21.15.49:0/2657583022 >> v2:172.21.15.183:3300/0 conn(0x7f2640155180 msgr2=0x7f26400c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 --2- 172.21.15.49:0/2657583022 >> v2:172.21.15.183:3300/0 conn(0x7f2640155180 0x7f26400c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.384+0000 7f263ffff640 1 -- 172.21.15.49:0/2657583022 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264015e3a0 con 0x7f264006b740 2024-05-09T08:58:15.385 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.385+0000 7f263ffff640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264006b740 0x7f26400c2730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f264006cef0 tx=0x7f263802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e14ba69ed8636c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.385+0000 7f263e7fc640 1 -- 172.21.15.49:0/2657583022 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638036070 con 0x7f264006b740 2024-05-09T08:58:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.385+0000 7f263e7fc640 1 -- 172.21.15.49:0/2657583022 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638031600 con 0x7f264006b740 2024-05-09T08:58:15.386 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.385+0000 7f263e7fc640 1 -- 172.21.15.49:0/2657583022 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380319d0 con 0x7f264006b740 2024-05-09T08:58:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 -- 172.21.15.49:0/2657583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264006b740 msgr2=0x7f26400c2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264006b740 0x7f26400c2730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f264006cef0 tx=0x7f263802a610 comp rx=0 tx=0).stop 2024-05-09T08:58:15.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 -- 172.21.15.49:0/2657583022 shutdown_connections 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f264006b740 0x7f26400c2730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 --2- 172.21.15.49:0/2657583022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f264006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 --2- 172.21.15.49:0/2657583022 >> v2:172.21.15.183:3300/0 conn(0x7f2640155180 0x7f26400c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 -- 172.21.15.49:0/2657583022 >> 172.21.15.49:0/2657583022 conn(0x7f26400c9650 msgr2=0x7f26400c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 -- 172.21.15.49:0/2657583022 shutdown_connections 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.386+0000 7f2645edd640 1 -- 172.21.15.49:0/2657583022 wait complete. 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.387+0000 7f2645edd640 1 Processor -- start 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.387+0000 7f2645edd640 1 -- start start 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264006b740 0x7f264014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f2640156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f26400c7550 con 0x7f2640155180 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f264015e960 con 0x7f264006b740 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f2645edd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f26400c32a0 con 0x7f26400c8f90 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48258/0 (socket says 172.21.15.49:48258) 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 -- 172.21.15.49:0/3384302789 learned_addr learned my addr 172.21.15.49:0/3384302789 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263effd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f2640156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263f7fe640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264006b740 0x7f264014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 -- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 msgr2=0x7f2640156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f2640156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 -- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264006b740 msgr2=0x7f264014e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264006b740 0x7f264014e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 -- 172.21.15.49:0/3384302789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26401f10e0 con 0x7f2640155180 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.388+0000 7f263ffff640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2638009f40 tx=0x7f2638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.389+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638036050 con 0x7f2640155180 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.389+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638042070 con 0x7f2640155180 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.389+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638009510 con 0x7f2640155180 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.389+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26401f13c0 con 0x7f2640155180 2024-05-09T08:58:15.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.389+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401f1980 con 0x7f2640155180 2024-05-09T08:58:15.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.390+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2640067c40 con 0x7f2640155180 2024-05-09T08:58:15.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.390+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2638007bd0 con 0x7f2640155180 2024-05-09T08:58:15.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.391+0000 7f2637fff640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:15.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.391+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5911+0+0 (secure 0 0 0) 0x7f26380abcc0 con 0x7f2640155180 2024-05-09T08:58:15.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.391+0000 7f263f7fe640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:15.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.391+0000 7f263f7fe640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 0x7f262006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:15.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.395+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2640067c40 con 0x7f2640155180 2024-05-09T08:58:15.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:15.591+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f26400cac50 con 0x7f2640155180 2024-05-09T08:58:16.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.299+0000 7f2637fff640 1 -- 172.21.15.49:0/3384302789 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v69) ==== 113+0+0 (secure 0 0 0) 0x7f26400cac50 con 0x7f2640155180 2024-05-09T08:58:16.298 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 msgr2=0x7f262006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 0x7f262006e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).stop 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 msgr2=0x7f2640157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2638009f40 tx=0x7f2638004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 shutdown_connections 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2640155180 0x7f2640157160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26400c8f90 0x7f2640156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 --2- 172.21.15.49:0/3384302789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f264006b740 0x7f264014e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 >> 172.21.15.49:0/3384302789 conn(0x7f26400c9650 msgr2=0x7f2640153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 shutdown_connections 2024-05-09T08:58:16.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.302+0000 7f2645edd640 1 -- 172.21.15.49:0/3384302789 wait complete. 2024-05-09T08:58:16.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-09T08:58:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.426+0000 7ff608142640 1 Processor -- start 2024-05-09T08:58:16.426 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.426+0000 7ff608142640 1 -- start start 2024-05-09T08:58:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff608142640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff600154f70 0x7ff6000c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.428 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff608142640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff6000c6b80 0x7ff600071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff608142640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6000c6fd0 con 0x7ff600154b90 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff608142640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff600072600 con 0x7ff600154f70 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff608142640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff6000728e0 con 0x7ff6000c6b80 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff6056dd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff600154f70 0x7ff6000c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.427+0000 7ff6056dd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff600154f70 0x7ff6000c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48268/0 (socket says 172.21.15.49:48268) 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 -- 172.21.15.49:0/2369529914 learned_addr learned my addr 172.21.15.49:0/2369529914 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6066df640 1 --2- 172.21.15.49:0/2369529914 >> v2:172.21.15.183:3300/0 conn(0x7ff6000c6b80 0x7ff600071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 -- 172.21.15.49:0/2369529914 >> v2:172.21.15.183:3300/0 conn(0x7ff6000c6b80 msgr2=0x7ff600071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 --2- 172.21.15.49:0/2369529914 >> v2:172.21.15.183:3300/0 conn(0x7ff6000c6b80 0x7ff600071820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 -- 172.21.15.49:0/2369529914 >> v1:172.21.15.183:6789/0 conn(0x7ff600154b90 legacy=0x7ff6000cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.429 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 -- 172.21.15.49:0/2369529914 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000c5880 con 0x7ff600154f70 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.428+0000 7ff6056dd640 1 --2- 172.21.15.49:0/2369529914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff6000c6640 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b4560 tx=0x7ff5f402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c83e27e1b074781 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff604edc640 1 -- 172.21.15.49:0/2369529914 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4037070 con 0x7ff600154f70 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff604edc640 1 -- 172.21.15.49:0/2369529914 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f402ae50 con 0x7ff600154f70 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff604edc640 1 -- 172.21.15.49:0/2369529914 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4032860 con 0x7ff600154f70 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 -- 172.21.15.49:0/2369529914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 msgr2=0x7ff6000c6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 --2- 172.21.15.49:0/2369529914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff6000c6640 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6000b4560 tx=0x7ff5f402a300 comp rx=0 tx=0).stop 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 -- 172.21.15.49:0/2369529914 shutdown_connections 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 --2- 172.21.15.49:0/2369529914 >> v2:172.21.15.183:3300/0 conn(0x7ff6000c6b80 0x7ff600071820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 --2- 172.21.15.49:0/2369529914 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff6000c6640 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 -- 172.21.15.49:0/2369529914 >> 172.21.15.49:0/2369529914 conn(0x7ff600074840 msgr2=0x7ff600074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:16.430 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 -- 172.21.15.49:0/2369529914 shutdown_connections 2024-05-09T08:58:16.431 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.429+0000 7ff608142640 1 -- 172.21.15.49:0/2369529914 wait complete. 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.430+0000 7ff608142640 1 Processor -- start 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 -- start start 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6000c6b80 0x7ff6001f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 0x7ff600156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.433 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000c6120 con 0x7ff600154f70 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000c5e40 con 0x7ff6000c6b80 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff608142640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff60015af40 con 0x7ff600154b90 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff6066df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff6066df640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48276/0 (socket says 172.21.15.49:48276) 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff6066df640 1 -- 172.21.15.49:0/3880676547 learned_addr learned my addr 172.21.15.49:0/3880676547 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff6056dd640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 0x7ff600156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.431+0000 7ff605ede640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6000c6b80 0x7ff6001f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 -- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 msgr2=0x7ff600156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 0x7ff600156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 -- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6000c6b80 msgr2=0x7ff6001f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6000c6b80 0x7ff6001f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:16.434 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 -- 172.21.15.49:0/3880676547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60015b850 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6056dd640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 0x7ff600156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff6066df640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0007e90 tx=0x7ff5f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0014420 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f0019070 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0014800 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6001f14a0 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.432+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6001555e0 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.433+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60004f660 con 0x7ff600154f70 2024-05-09T08:58:16.435 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.434+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5f00149a0 con 0x7ff600154f70 2024-05-09T08:58:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.434+0000 7ff5fe7fc640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 0x7ff5ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.434+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5886+0+0 (secure 0 0 0) 0x7ff5f008b7b0 con 0x7ff600154f70 2024-05-09T08:58:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.434+0000 7ff605ede640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 0x7ff5ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:16.436 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.435+0000 7ff605ede640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 0x7ff5ec06e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007fd0 tx=0x7ff5f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:16.437 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.438+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff60015b850 con 0x7ff600154f70 2024-05-09T08:58:16.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:16.634+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7ff600074e70 con 0x7ff600154f70 2024-05-09T08:58:17.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.314+0000 7ff5fe7fc640 1 -- 172.21.15.49:0/3880676547 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v70) ==== 133+0+0 (secure 0 0 0) 0x7ff5f005a020 con 0x7ff600154f70 2024-05-09T08:58:17.313 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-09T08:58:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 msgr2=0x7ff5ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 0x7ff5ec06e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8007fd0 tx=0x7ff5f8009040 comp rx=0 tx=0).stop 2024-05-09T08:58:17.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 msgr2=0x7ff60014cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0007e90 tx=0x7ff5f000e4b0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 shutdown_connections 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5ec06be30 0x7ff5ec06e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff600154f70 0x7ff60014cbf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff600154b90 0x7ff600156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 --2- 172.21.15.49:0/3880676547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6000c6b80 0x7ff6001f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 >> 172.21.15.49:0/3880676547 conn(0x7ff600074840 msgr2=0x7ff600159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 shutdown_connections 2024-05-09T08:58:17.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.317+0000 7ff608142640 1 -- 172.21.15.49:0/3880676547 wait complete. 2024-05-09T08:58:17.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-09T08:58:17.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.444+0000 7fd9a1693640 1 Processor -- start 2024-05-09T08:58:17.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.444+0000 7fd9a1693640 1 -- start start 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd9a1693640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd99c152e30 0x7fd99c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd9a1693640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd99c072670 0x7fd99c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd9a1693640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd99c159b10 con 0x7fd99c0c7130 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd9a1693640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd99c159270 con 0x7fd99c072670 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd9a1693640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd99c159550 con 0x7fd99c152e30 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd99b7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd99c072670 0x7fd99c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd99a7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd99c0c7130 0x7fd99c0c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51196/0 (socket says 172.21.15.49:51196) 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd99a7fc640 1 -- 172.21.15.49:0/2788958538 learned_addr learned my addr 172.21.15.49:0/2788958538 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.445+0000 7fd99affd640 1 --2- 172.21.15.49:0/2788958538 >> v2:172.21.15.183:3300/0 conn(0x7fd99c152e30 0x7fd99c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2788958538 >> v2:172.21.15.183:3300/0 conn(0x7fd99c152e30 msgr2=0x7fd99c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:17.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd99b7fe640 1 --2- 172.21.15.49:0/2788958538 >> v2:172.21.15.183:3300/0 conn(0x7fd99c152e30 0x7fd99c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2788958538 >> v1:172.21.15.49:6789/0 conn(0x7fd99c0c7130 legacy=0x7fd99c0c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2788958538 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c159830 con 0x7fd99c072670 2024-05-09T08:58:17.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd999ffb640 1 -- 172.21.15.49:0/2788958538 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769880499 0 0) 0x7fd99c159b10 con 0x7fd99c0c7130 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd99b7fe640 1 --2- 172.21.15.49:0/2788958538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c072670 0x7fd99c0c3790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0cce70 tx=0x7fd98c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f01e5a58f47e85 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.446+0000 7fd999ffb640 1 -- 172.21.15.49:0/2788958538 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c037070 con 0x7fd99c072670 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd999ffb640 1 -- 172.21.15.49:0/2788958538 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c032440 con 0x7fd99c072670 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd999ffb640 1 -- 172.21.15.49:0/2788958538 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c032810 con 0x7fd99c072670 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 -- 172.21.15.49:0/2788958538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c072670 msgr2=0x7fd99c0c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2788958538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c072670 0x7fd99c0c3790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0cce70 tx=0x7fd98c02a1a0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 -- 172.21.15.49:0/2788958538 shutdown_connections 2024-05-09T08:58:17.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2788958538 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c072670 0x7fd99c0c3790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2788958538 >> v2:172.21.15.183:3300/0 conn(0x7fd99c152e30 0x7fd99c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 -- 172.21.15.49:0/2788958538 >> 172.21.15.49:0/2788958538 conn(0x7fd99c0cc290 msgr2=0x7fd99c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:17.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 -- 172.21.15.49:0/2788958538 shutdown_connections 2024-05-09T08:58:17.448 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.447+0000 7fd9a1693640 1 -- 172.21.15.49:0/2788958538 wait complete. 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.448+0000 7fd9a1693640 1 Processor -- start 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.448+0000 7fd9a1693640 1 -- start start 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd99c072670 0x7fd99c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 0x7fd99c1f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c0c6620 con 0x7fd99c152e30 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c0c59f0 con 0x7fd99c0c7130 2024-05-09T08:58:17.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd9a1693640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c06b4b0 con 0x7fd99c072670 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd99b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd99b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48288/0 (socket says 172.21.15.49:48288) 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2920575593 learned_addr learned my addr 172.21.15.49:0/2920575593 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.449+0000 7fd99affd640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd99c072670 0x7fd99c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99a7fc640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 0x7fd99c1f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd99c072670 msgr2=0x7fd99c157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd99c072670 0x7fd99c157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 msgr2=0x7fd99c1f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 0x7fd99c1f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 -- 172.21.15.49:0/2920575593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c158ad0 con 0x7fd99c152e30 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99a7fc640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 0x7fd99c1f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd99b7fe640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd98c035040 tx=0x7fd98c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:17.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c037040 con 0x7fd99c152e30 2024-05-09T08:58:17.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c042070 con 0x7fd99c152e30 2024-05-09T08:58:17.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1518b0 con 0x7fd99c152e30 2024-05-09T08:58:17.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c030b30 con 0x7fd99c152e30 2024-05-09T08:58:17.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.450+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1f0710 con 0x7fd99c152e30 2024-05-09T08:58:17.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.451+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c0c6980 con 0x7fd99c152e30 2024-05-09T08:58:17.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.451+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd98c030cd0 con 0x7fd99c152e30 2024-05-09T08:58:17.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.452+0000 7fd9937fe640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 0x7fd97c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:17.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.452+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5886+0+0 (secure 0 0 0) 0x7fd98c0aba60 con 0x7fd99c152e30 2024-05-09T08:58:17.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.452+0000 7fd99affd640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 0x7fd97c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:17.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.453+0000 7fd99affd640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 0x7fd97c06e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd994009a60 tx=0x7fd994003990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:17.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.456+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd99c0c6980 con 0x7fd99c152e30 2024-05-09T08:58:17.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:17.651+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7fd99c04f660 con 0x7fd99c152e30 2024-05-09T08:58:18.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.315+0000 7fd9937fe640 1 -- 172.21.15.49:0/2920575593 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v71) ==== 113+0+0 (secure 0 0 0) 0x7fd98c032e20 con 0x7fd99c152e30 2024-05-09T08:58:18.315 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-05-09T08:58:18.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 msgr2=0x7fd97c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 0x7fd97c06e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd994009a60 tx=0x7fd994003990 comp rx=0 tx=0).stop 2024-05-09T08:58:18.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 msgr2=0x7fd99c1e0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd98c035040 tx=0x7fd98c0046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 shutdown_connections 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd97c06be30 0x7fd97c06e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd99c152e30 0x7fd99c1e0370 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c0c7130 0x7fd99c1f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 --2- 172.21.15.49:0/2920575593 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd99c072670 0x7fd99c157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 >> 172.21.15.49:0/2920575593 conn(0x7fd99c0cc290 msgr2=0x7fd99c151d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 shutdown_connections 2024-05-09T08:58:18.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.319+0000 7fd9a1693640 1 -- 172.21.15.49:0/2920575593 wait complete. 2024-05-09T08:58:18.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-09T08:58:18.443 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.444+0000 7f5b5a8d4640 1 Processor -- start 2024-05-09T08:58:18.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.444+0000 7f5b5a8d4640 1 -- start start 2024-05-09T08:58:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b5a8d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154f70 0x7f5b540c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.445 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b5a8d4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b540c6b80 0x7f5b54071820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b5a8d4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b540c6fd0 con 0x7f5b54154b90 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b5a8d4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5b54072600 con 0x7f5b540c6b80 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b5a8d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540728e0 con 0x7f5b54154f70 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b540c6b80 0x7f5b54071820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5b540c6b80 0x7f5b54071820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48300/0 (socket says 172.21.15.49:48300) 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 -- 172.21.15.49:0/698511817 learned_addr learned my addr 172.21.15.49:0/698511817 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b4bfff640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154f70 0x7f5b540c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 -- 172.21.15.49:0/698511817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154f70 msgr2=0x7f5b540c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154f70 0x7f5b540c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 -- 172.21.15.49:0/698511817 >> v1:172.21.15.183:6789/0 conn(0x7f5b54154b90 legacy=0x7f5b540cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.445+0000 7f5b58e71640 1 -- 172.21.15.49:0/698511817 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540c5880 con 0x7f5b540c6b80 2024-05-09T08:58:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.446+0000 7f5b58e71640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b540c6b80 0x7f5b54071820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009e50 tx=0x7f5b4c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7091de472eea513c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.446+0000 7f5b537fe640 1 -- 172.21.15.49:0/698511817 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036070 con 0x7f5b540c6b80 2024-05-09T08:58:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.446+0000 7f5b537fe640 1 -- 172.21.15.49:0/698511817 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c031600 con 0x7f5b540c6b80 2024-05-09T08:58:18.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.446+0000 7f5b537fe640 1 -- 172.21.15.49:0/698511817 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0319d0 con 0x7f5b540c6b80 2024-05-09T08:58:18.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/698511817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b540c6b80 msgr2=0x7f5b54071820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b540c6b80 0x7f5b54071820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009e50 tx=0x7f5b4c02a7a0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.449 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/698511817 shutdown_connections 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b540c6b80 0x7f5b54071820 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/698511817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154f70 0x7f5b540c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/698511817 >> 172.21.15.49:0/698511817 conn(0x7f5b54074840 msgr2=0x7f5b54074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/698511817 shutdown_connections 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.447+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/698511817 wait complete. 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.448+0000 7f5b5a8d4640 1 Processor -- start 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.448+0000 7f5b5a8d4640 1 -- start start 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b540c6b80 0x7f5b5414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154b90 0x7f5b540c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540c6120 con 0x7f5b54154f70 2024-05-09T08:58:18.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b540c5e40 con 0x7f5b540c6b80 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b5a8d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5415af40 con 0x7f5b54154b90 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48302/0 (socket says 172.21.15.49:48302) 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 -- 172.21.15.49:0/3738501009 learned_addr learned my addr 172.21.15.49:0/3738501009 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b4bfff640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154b90 0x7f5b540c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b53fff640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b540c6b80 0x7f5b5414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 -- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154b90 msgr2=0x7f5b540c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154b90 0x7f5b540c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 -- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b540c6b80 msgr2=0x7f5b5414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b540c6b80 0x7f5b5414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.449+0000 7f5b58e71640 1 -- 172.21.15.49:0/3738501009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540ceea0 con 0x7f5b54154f70 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b58e71640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009f70 tx=0x7f5b4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:18.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c036050 con 0x7f5b54154f70 2024-05-09T08:58:18.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c042070 con 0x7f5b54154f70 2024-05-09T08:58:18.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b540cf9e0 con 0x7f5b54154f70 2024-05-09T08:58:18.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c009510 con 0x7f5b54154f70 2024-05-09T08:58:18.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.450+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b540cf140 con 0x7f5b54154f70 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.451+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b5404f660 con 0x7f5b54154f70 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.451+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b4c007bf0 con 0x7f5b54154f70 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.452+0000 7f5b517fa640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 0x7f5b3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.452+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5886+0+0 (secure 0 0 0) 0x7f5b4c0ab550 con 0x7f5b54154f70 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.452+0000 7f5b53fff640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 0x7f5b3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:18.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.453+0000 7f5b53fff640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 0x7f5b3406e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b44001780 tx=0x7f5b44009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:18.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.456+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b4c04b020 con 0x7f5b54154f70 2024-05-09T08:58:18.655 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:18.655+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7f5b540d12a0 con 0x7f5b54154f70 2024-05-09T08:58:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.321+0000 7f5b517fa640 1 -- 172.21.15.49:0/3738501009 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v72) ==== 133+0+0 (secure 0 0 0) 0x7f5b4c079800 con 0x7f5b54154f70 2024-05-09T08:58:19.390 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-09T08:58:19.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 msgr2=0x7f5b3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 0x7f5b3406e320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b44001780 tx=0x7f5b44009040 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 msgr2=0x7f5b5414cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009f70 tx=0x7f5b4c004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 shutdown_connections 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5b3406be60 0x7f5b3406e320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5b54154f70 0x7f5b5414cb10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b54154b90 0x7f5b540c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 --2- 172.21.15.49:0/3738501009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b540c6b80 0x7f5b5414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 >> 172.21.15.49:0/3738501009 conn(0x7f5b54074840 msgr2=0x7f5b540b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 shutdown_connections 2024-05-09T08:58:19.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.325+0000 7f5b5a8d4640 1 -- 172.21.15.49:0/3738501009 wait complete. 2024-05-09T08:58:19.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-09T08:58:19.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.508+0000 7ff8a411b640 1 Processor -- start 2024-05-09T08:58:19.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.508+0000 7ff8a411b640 1 -- start start 2024-05-09T08:58:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff89c151be0 0x7ff89c0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff89c15af60 0x7ff89c0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c0c71d0 0x7ff89c06dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff89c0c5c70 con 0x7ff89c151be0 2024-05-09T08:58:19.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7ff89c0c4b10 con 0x7ff89c15af60 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a411b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff89c0c4da0 con 0x7ff89c0c71d0 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff89c151be0 0x7ff89c0c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff89c151be0 0x7ff89c0c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48314/0 (socket says 172.21.15.49:48314) 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3070836520 learned_addr learned my addr 172.21.15.49:0/3070836520 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a16b6640 1 --2- 172.21.15.49:0/3070836520 >> v2:172.21.15.183:3300/0 conn(0x7ff89c15af60 0x7ff89c0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a26b8640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c0c71d0 0x7ff89c06dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3070836520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c0c71d0 msgr2=0x7ff89c06dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c0c71d0 0x7ff89c06dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3070836520 >> v2:172.21.15.183:3300/0 conn(0x7ff89c15af60 msgr2=0x7ff89c0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3070836520 >> v2:172.21.15.183:3300/0 conn(0x7ff89c15af60 0x7ff89c0c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.509+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3070836520 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c0c5060 con 0x7ff89c151be0 2024-05-09T08:58:19.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.510+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c151be0 0x7ff89c0c45d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff89c15bfe0 tx=0x7ff89802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598623bd05116e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.510+0000 7ff8a0eb5640 1 -- 172.21.15.49:0/3070836520 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898037070 con 0x7ff89c151be0 2024-05-09T08:58:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.510+0000 7ff8a0eb5640 1 -- 172.21.15.49:0/3070836520 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff898032440 con 0x7ff89c151be0 2024-05-09T08:58:19.511 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.510+0000 7ff8a0eb5640 1 -- 172.21.15.49:0/3070836520 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898032810 con 0x7ff89c151be0 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 -- 172.21.15.49:0/3070836520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c151be0 msgr2=0x7ff89c0c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c151be0 0x7ff89c0c45d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff89c15bfe0 tx=0x7ff89802a270 comp rx=0 tx=0).stop 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 -- 172.21.15.49:0/3070836520 shutdown_connections 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c0c71d0 0x7ff89c06dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3070836520 >> v2:172.21.15.183:3300/0 conn(0x7ff89c15af60 0x7ff89c0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3070836520 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c151be0 0x7ff89c0c45d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 -- 172.21.15.49:0/3070836520 >> 172.21.15.49:0/3070836520 conn(0x7ff89c0cc290 msgr2=0x7ff89c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:19.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 -- 172.21.15.49:0/3070836520 shutdown_connections 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.511+0000 7ff8a411b640 1 -- 172.21.15.49:0/3070836520 wait complete. 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.512+0000 7ff8a411b640 1 Processor -- start 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.512+0000 7ff8a411b640 1 -- start start 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c151be0 0x7ff89c0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff89c15af60 0x7ff89c147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c0c5900 con 0x7ff89c0c71d0 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff89c0c5620 con 0x7ff89c15af60 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff89c151fc0 con 0x7ff89c151be0 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48316/0 (socket says 172.21.15.49:48316) 2024-05-09T08:58:19.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3273624271 learned_addr learned my addr 172.21.15.49:0/3273624271 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a16b6640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c151be0 0x7ff89c0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a26b8640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff89c15af60 0x7ff89c147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c151be0 msgr2=0x7ff89c0708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c151be0 0x7ff89c0708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff89c15af60 msgr2=0x7ff89c147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff89c15af60 0x7ff89c147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 -- 172.21.15.49:0/3273624271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89c072000 con 0x7ff89c0c71d0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a1eb7640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff8980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898041020 con 0x7ff89c0c71d0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8980306c0 con 0x7ff89c0c71d0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89c148d40 con 0x7ff89c0c71d0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.513+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898037040 con 0x7ff89c0c71d0 2024-05-09T08:58:19.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.514+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff89c149300 con 0x7ff89c0c71d0 2024-05-09T08:58:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.514+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89c04f660 con 0x7ff89c0c71d0 2024-05-09T08:58:19.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.515+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff898044020 con 0x7ff89c0c71d0 2024-05-09T08:58:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.515+0000 7ff8927fc640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 0x7ff88006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.516+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5886+0+0 (secure 0 0 0) 0x7ff8980ab620 con 0x7ff89c0c71d0 2024-05-09T08:58:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.516+0000 7ff8a16b6640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 0x7ff88006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:19.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.516+0000 7ff8a16b6640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 0x7ff88006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1e5cd0 tx=0x7ff88c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:19.519 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.520+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff89804b030 con 0x7ff89c0c71d0 2024-05-09T08:58:19.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:19.714+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7ff89c0c79a0 con 0x7ff89c0c71d0 2024-05-09T08:58:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.327+0000 7ff8927fc640 1 -- 172.21.15.49:0/3273624271 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v73) ==== 113+0+0 (secure 0 0 0) 0x7ff898079fb0 con 0x7ff89c0c71d0 2024-05-09T08:58:20.326 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-05-09T08:58:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 msgr2=0x7ff88006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 0x7ff88006e320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff89c1e5cd0 tx=0x7ff88c008040 comp rx=0 tx=0).stop 2024-05-09T08:58:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 msgr2=0x7ff89c1e5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff898000c00 tx=0x7ff8980046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 shutdown_connections 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff88006be60 0x7ff88006e320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff89c15af60 0x7ff89c147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff89c151be0 0x7ff89c0708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 --2- 172.21.15.49:0/3273624271 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff89c0c71d0 0x7ff89c1e5420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 >> 172.21.15.49:0/3273624271 conn(0x7ff89c0cc290 msgr2=0x7ff89c0c7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 shutdown_connections 2024-05-09T08:58:20.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.330+0000 7ff8a411b640 1 -- 172.21.15.49:0/3273624271 wait complete. 2024-05-09T08:58:20.344 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-09T08:58:20.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.456+0000 7ff5c6c72640 1 Processor -- start 2024-05-09T08:58:20.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.456+0000 7ff5c6c72640 1 -- start start 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.456+0000 7ff5c6c72640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff5c0154b90 0x7ff5c00cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c6c72640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00c6d50 0x7ff5c00cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c6c72640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c0072600 con 0x7ff5c0154f70 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c6c72640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c00728e0 con 0x7ff5c0154b90 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c6c72640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00c5880 con 0x7ff5c00c6d50 2024-05-09T08:58:20.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff5c0154b90 0x7ff5c00cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5bffff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff5c0154f70 0x7ff5c00c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51206/0 (socket says 172.21.15.49:51206) 2024-05-09T08:58:20.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7ff5c0154b90 0x7ff5c00cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48326/0 (socket says 172.21.15.49:48326) 2024-05-09T08:58:20.457 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5bffff640 1 -- 172.21.15.49:0/121755464 learned_addr learned my addr 172.21.15.49:0/121755464 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:20.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 -- 172.21.15.49:0/121755464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00c6d50 msgr2=0x7ff5c00cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:20.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c520f640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00c6d50 0x7ff5c00cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00c6d50 0x7ff5c00cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 -- 172.21.15.49:0/121755464 >> v1:172.21.15.49:6789/0 conn(0x7ff5c0154f70 legacy=0x7ff5c00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5c4a0e640 1 -- 172.21.15.49:0/121755464 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00c5b60 con 0x7ff5c0154b90 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.457+0000 7ff5bf7fe640 1 -- 172.21.15.49:0/121755464 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236159721 0 0) 0x7ff5c0072600 con 0x7ff5c0154f70 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c00cb280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9850765533d1d9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5bf7fe640 1 -- 172.21.15.49:0/121755464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007740 con 0x7ff5c0154b90 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5bf7fe640 1 -- 172.21.15.49:0/121755464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b00078e0 con 0x7ff5c0154b90 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5bf7fe640 1 -- 172.21.15.49:0/121755464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0007cb0 con 0x7ff5c0154b90 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 -- 172.21.15.49:0/121755464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 msgr2=0x7ff5c00cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c00cb280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009d90 tx=0x7ff5b002a6f0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 -- 172.21.15.49:0/121755464 shutdown_connections 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00c6d50 0x7ff5c00cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/121755464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c00cb280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.458+0000 7ff5c6c72640 1 -- 172.21.15.49:0/121755464 >> 172.21.15.49:0/121755464 conn(0x7ff5c0074840 msgr2=0x7ff5c0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.459+0000 7ff5c6c72640 1 -- 172.21.15.49:0/121755464 shutdown_connections 2024-05-09T08:58:20.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.459+0000 7ff5c6c72640 1 -- 172.21.15.49:0/121755464 wait complete. 2024-05-09T08:58:20.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.460+0000 7ff5c6c72640 1 Processor -- start 2024-05-09T08:58:20.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.460+0000 7ff5c6c72640 1 -- start start 2024-05-09T08:58:20.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 0x7ff5c01501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.464 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 0x7ff5c014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c015af40 con 0x7ff5c0154b90 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00c6120 con 0x7ff5c00c6d50 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c0071820 con 0x7ff5c0154f70 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48336/0 (socket says 172.21.15.49:48336) 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 -- 172.21.15.49:0/1823472288 learned_addr learned my addr 172.21.15.49:0/1823472288 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:20.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c520f640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 0x7ff5c014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 0x7ff5c01501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 -- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 msgr2=0x7ff5c014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 0x7ff5c014ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 -- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 msgr2=0x7ff5c01501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 0x7ff5c01501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 -- 172.21.15.49:0/1823472288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00b5230 con 0x7ff5c0154b90 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c520f640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 0x7ff5c014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 0x7ff5c01501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:20.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bffff640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c460 tx=0x7ff5a800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8018070 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a8007500 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8004a20 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c00ca3c0 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.461+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c00c9a60 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.463+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00b67d0 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.463+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5a8004bc0 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.463+0000 7ff5bd7fa640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 0x7ff5a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.464+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5886+0+0 (secure 0 0 0) 0x7ff5a808a2e0 con 0x7ff5c0154b90 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.464+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 0x7ff5a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:20.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.464+0000 7ff5c4a0e640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 0x7ff5a006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0034000 tx=0x7ff5b002f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:20.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.468+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff5c00b67d0 con 0x7ff5c0154b90 2024-05-09T08:58:20.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:20.663+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7ff5c004f660 con 0x7ff5c0154b90 2024-05-09T08:58:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.332+0000 7ff5bd7fa640 1 -- 172.21.15.49:0/1823472288 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v74) ==== 133+0+0 (secure 0 0 0) 0x7ff5a8058b50 con 0x7ff5c0154b90 2024-05-09T08:58:21.332 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 msgr2=0x7ff5a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 0x7ff5a006e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0034000 tx=0x7ff5b002f000 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 msgr2=0x7ff5c014a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c460 tx=0x7ff5a800c920 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 shutdown_connections 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff5a006be30 0x7ff5a006e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c0154f70 0x7ff5c014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff5c0154b90 0x7ff5c014a380 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 --2- 172.21.15.49:0/1823472288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c00c6d50 0x7ff5c01501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 >> 172.21.15.49:0/1823472288 conn(0x7ff5c0074840 msgr2=0x7ff5c00b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:21.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 shutdown_connections 2024-05-09T08:58:21.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.335+0000 7ff5c6c72640 1 -- 172.21.15.49:0/1823472288 wait complete. 2024-05-09T08:58:21.347 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-09T08:58:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.455+0000 7f0457742640 1 Processor -- start 2024-05-09T08:58:21.455 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.455+0000 7f0457742640 1 -- start start 2024-05-09T08:58:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f0457742640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04501551d0 0x7f045015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.458 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f0457742640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f045015e800 0x7f045006f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f0457742640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0450071ef0 con 0x7f0450154df0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f0457742640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f045015ed50 con 0x7f045015e800 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f0457742640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f045015f030 con 0x7f04501551d0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f04554de640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f0450154df0 0x7f045015c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:51212/0 (socket says 172.21.15.49:51212) 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.456+0000 7f04554de640 1 -- 172.21.15.49:0/3272568068 learned_addr learned my addr 172.21.15.49:0/3272568068 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f0455cdf640 1 --2- 172.21.15.49:0/3272568068 >> v2:172.21.15.183:3300/0 conn(0x7f045015e800 0x7f045006f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f0454cdd640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04501551d0 0x7f045015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87359887 0 0) 0x7f0450071ef0 con 0x7f0450154df0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0438003520 con 0x7f0450154df0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3350161258 0 0) 0x7f0438003520 con 0x7f0450154df0 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04501551d0 msgr2=0x7f045015e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:21.459 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04501551d0 0x7f045015e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 >> v2:172.21.15.183:3300/0 conn(0x7f045015e800 msgr2=0x7f045006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 --2- 172.21.15.49:0/3272568068 >> v2:172.21.15.183:3300/0 conn(0x7f045015e800 0x7f045006f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045015aec0 con 0x7f0450154df0 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f0448001670 con 0x7f0450154df0 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 >> v1:172.21.15.49:6789/0 conn(0x7f0450154df0 legacy=0x7f045015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04380039b0 0x7f0438003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438004800 0x7f0438004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04380051c0 0x7f043800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f045015ed50 con 0x7f04380039b0 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f045015f030 con 0x7f0438004800 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f043800bf00 con 0x7f04380051c0 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04380039b0 0x7f0438003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.460 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f0454cdd640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438004800 0x7f0438004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f0455cdf640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04380051c0 0x7f043800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 -- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04380051c0 msgr2=0x7f043800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f04380051c0 0x7f043800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 -- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438004800 msgr2=0x7f0438004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0438004800 0x7f0438004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.457+0000 7f04554de640 1 -- 172.21.15.49:0/3272568068 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0438003520 con 0x7f04380039b0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.458+0000 7f04554de640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04380039b0 0x7f0438003df0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f044800c240 tx=0x7f0438005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35445741de68b904 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.458+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044802e070 con 0x7f04380039b0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.458+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0448005af0 con 0x7f04380039b0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.458+0000 7f044ffff640 1 -- 172.21.15.49:0/3272568068 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448007d30 con 0x7f04380039b0 2024-05-09T08:58:21.461 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 -- 172.21.15.49:0/3272568068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04380039b0 msgr2=0x7f0438003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 --2- 172.21.15.49:0/3272568068 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04380039b0 0x7f0438003df0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f044800c240 tx=0x7f0438005830 comp rx=0 tx=0).stop 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0455cdf640 1 -- 172.21.15.49:0/3272568068 reap_dead start 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 -- 172.21.15.49:0/3272568068 shutdown_connections 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 -- 172.21.15.49:0/3272568068 >> 172.21.15.49:0/3272568068 conn(0x7f045006b200 msgr2=0x7f045006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 -- 172.21.15.49:0/3272568068 shutdown_connections 2024-05-09T08:58:21.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.459+0000 7f0457742640 1 -- 172.21.15.49:0/3272568068 wait complete. 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.460+0000 7f0457742640 1 Processor -- start 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.460+0000 7f0457742640 1 -- start start 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 0x7f04500c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f045015dd70 0x7f0450159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f04500c7e80 con 0x7f04500ccc20 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f045015b480 con 0x7f04380039b0 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0457742640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f04500c8000 con 0x7f045015dd70 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.465 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48350/0 (socket says 172.21.15.49:48350) 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 -- 172.21.15.49:0/2085636843 learned_addr learned my addr 172.21.15.49:0/2085636843 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0454cdd640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f045015dd70 0x7f0450159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f04554de640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 0x7f04500c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 -- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f045015dd70 msgr2=0x7f0450159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f045015dd70 0x7f0450159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 -- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 msgr2=0x7f04500c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 0x7f04500c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 -- 172.21.15.49:0/2085636843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04500c7810 con 0x7f04500ccc20 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f04554de640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 0x7f04500c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f0455cdf640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0450072560 tx=0x7f044000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044000fcb0 con 0x7f04500ccc20 2024-05-09T08:58:21.466 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.461+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044001d070 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.462+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440015670 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.462+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04501517b0 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.462+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501f3470 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.463+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045004f660 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.463+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0440015810 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.464+0000 7f044dffb640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 0x7f043c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.464+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5886+0+0 (secure 0 0 0) 0x7f044008b6f0 con 0x7f04500ccc20 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.464+0000 7f04554de640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 0x7f043c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:21.467 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.465+0000 7f04554de640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 0x7f043c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0448033000 tx=0x7f0448009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:21.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.468+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0440010040 con 0x7f04500ccc20 2024-05-09T08:58:21.662 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:21.663+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f0450153660 con 0x7f04500ccc20 2024-05-09T08:58:22.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.339+0000 7f044dffb640 1 -- 172.21.15.49:0/2085636843 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v75) ==== 113+0+0 (secure 0 0 0) 0x7f0440059ff0 con 0x7f04500ccc20 2024-05-09T08:58:22.339 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-05-09T08:58:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 msgr2=0x7f043c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 0x7f043c06e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0448033000 tx=0x7f0448009030 comp rx=0 tx=0).stop 2024-05-09T08:58:22.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 msgr2=0x7f045015a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0450072560 tx=0x7f044000f1b0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 shutdown_connections 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f043c06be30 0x7f043c06e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f04500ccc20 0x7f045015a200 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f045015dd70 0x7f0450159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 --2- 172.21.15.49:0/2085636843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04380039b0 0x7f04500c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 >> 172.21.15.49:0/2085636843 conn(0x7f045006b200 msgr2=0x7f0450074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:22.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 shutdown_connections 2024-05-09T08:58:22.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.342+0000 7f0457742640 1 -- 172.21.15.49:0/2085636843 wait complete. 2024-05-09T08:58:22.357 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-09T08:58:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.470+0000 7fd187f02640 1 Processor -- start 2024-05-09T08:58:22.469 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.470+0000 7fd187f02640 1 -- start start 2024-05-09T08:58:22.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd187f02640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 0x7fd1800c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd187f02640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd18015af60 0x7fd1800c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd187f02640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1800c4b10 con 0x7fd1800c71d0 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd187f02640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd1800c7600 con 0x7fd18015af60 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd187f02640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd1800c5c70 con 0x7fd180151be0 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd18015af60 0x7fd1800c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd18015af60 0x7fd1800c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48352/0 (socket says 172.21.15.49:48352) 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 -- 172.21.15.49:0/3131785368 learned_addr learned my addr 172.21.15.49:0/3131785368 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 -- 172.21.15.49:0/3131785368 >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 msgr2=0x7fd1800c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd185c9e640 1 --2- 172.21.15.49:0/3131785368 >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 0x7fd1800c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 --2- 172.21.15.49:0/3131785368 >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 0x7fd1800c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 -- 172.21.15.49:0/3131785368 >> v1:172.21.15.183:6789/0 conn(0x7fd1800c71d0 legacy=0x7fd18006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd18549d640 1 -- 172.21.15.49:0/3131785368 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1800c4da0 con 0x7fd18015af60 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.471+0000 7fd185c9e640 1 --2- 172.21.15.49:0/3131785368 >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 0x7fd1800c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.472+0000 7fd18549d640 1 --2- 172.21.15.49:0/3131785368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18015af60 0x7fd1800c6c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd18006d1a0 tx=0x7fd17002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba5a23c3fe63900 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:22.472 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.472+0000 7fd184c9c640 1 -- 172.21.15.49:0/3131785368 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170037070 con 0x7fd18015af60 2024-05-09T08:58:22.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.472+0000 7fd184c9c640 1 -- 172.21.15.49:0/3131785368 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd170032440 con 0x7fd18015af60 2024-05-09T08:58:22.473 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.472+0000 7fd184c9c640 1 -- 172.21.15.49:0/3131785368 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170032810 con 0x7fd18015af60 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 -- 172.21.15.49:0/3131785368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18015af60 msgr2=0x7fd1800c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 --2- 172.21.15.49:0/3131785368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18015af60 0x7fd1800c6c90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd18006d1a0 tx=0x7fd17002a2c0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 -- 172.21.15.49:0/3131785368 shutdown_connections 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 --2- 172.21.15.49:0/3131785368 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd18015af60 0x7fd1800c6c90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 --2- 172.21.15.49:0/3131785368 >> v2:172.21.15.183:3300/0 conn(0x7fd180151be0 0x7fd1800c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 -- 172.21.15.49:0/3131785368 >> 172.21.15.49:0/3131785368 conn(0x7fd1800cc290 msgr2=0x7fd1800cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 -- 172.21.15.49:0/3131785368 shutdown_connections 2024-05-09T08:58:22.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.473+0000 7fd187f02640 1 -- 172.21.15.49:0/3131785368 wait complete. 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.474+0000 7fd187f02640 1 Processor -- start 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.474+0000 7fd187f02640 1 -- start start 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1800c71d0 0x7fd180158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18015af60 0x7fd180150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800c5620 con 0x7fd180151be0 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1800c5340 con 0x7fd18015af60 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd1800c5900 con 0x7fd1800c71d0 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:48356/0 (socket says 172.21.15.49:48356) 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 -- 172.21.15.49:0/2122762092 learned_addr learned my addr 172.21.15.49:0/2122762092 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd185c9e640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1800c71d0 0x7fd180158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 -- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1800c71d0 msgr2=0x7fd180158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18649f640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18015af60 0x7fd180150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1800c71d0 0x7fd180158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 -- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18015af60 msgr2=0x7fd180150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18015af60 0x7fd180150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:22.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 -- 172.21.15.49:0/2122762092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1801f29c0 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd18549d640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170041020 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd170030970 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.475+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1801efaf0 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.476+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170037040 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.476+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1801f00b0 con 0x7fd180151be0 2024-05-09T08:58:22.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.476+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18004f660 con 0x7fd180151be0 2024-05-09T08:58:22.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.477+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd170044020 con 0x7fd180151be0 2024-05-09T08:58:22.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.477+0000 7fd1767fc640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:22.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.478+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5886+0+0 (secure 0 0 0) 0x7fd1700ab950 con 0x7fd180151be0 2024-05-09T08:58:22.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.478+0000 7fd185c9e640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:22.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.478+0000 7fd185c9e640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 0x7fd16406e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17c008850 tx=0x7fd17c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:22.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.482+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd17004b030 con 0x7fd180151be0 2024-05-09T08:58:22.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:22.676+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7fd180075950 con 0x7fd180151be0 2024-05-09T08:58:23.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.345+0000 7fd1767fc640 1 -- 172.21.15.49:0/2122762092 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v76) ==== 133+0+0 (secure 0 0 0) 0x7fd17007a250 con 0x7fd180151be0 2024-05-09T08:58:23.345 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-09T08:58:23.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 msgr2=0x7fd16406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 0x7fd16406e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd17c008850 tx=0x7fd17c008040 comp rx=0 tx=0).stop 2024-05-09T08:58:23.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 msgr2=0x7fd1801f2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd1700046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 shutdown_connections 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd16406be30 0x7fd16406e2f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18015af60 0x7fd180150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd180151be0 0x7fd1801f2480 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 --2- 172.21.15.49:0/2122762092 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd1800c71d0 0x7fd180158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 >> 172.21.15.49:0/2122762092 conn(0x7fd1800cc290 msgr2=0x7fd1800caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 shutdown_connections 2024-05-09T08:58:23.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.349+0000 7fd187f02640 1 -- 172.21.15.49:0/2122762092 wait complete. 2024-05-09T08:58:23.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-09T08:58:23.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.477+0000 7f53ae1c1640 1 Processor -- start 2024-05-09T08:58:23.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.477+0000 7f53ae1c1640 1 -- start start 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53ae1c1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8154df0 0x7f53a815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53ae1c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80c7e80 0x7f53a806f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53ae1c1640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a815c700 con 0x7f53a81551d0 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53ae1c1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f53a8071ef0 con 0x7f53a8154df0 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53ae1c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53a815ed50 con 0x7f53a80c7e80 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8154df0 0x7f53a815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f53a8154df0 0x7f53a815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45272/0 (socket says 172.21.15.49:45272) 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a6ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f53a81551d0 0x7f53a815e2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47520/0 (socket says 172.21.15.49:47520) 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 -- 172.21.15.49:0/692627119 learned_addr learned my addr 172.21.15.49:0/692627119 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:23.478 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a7fff640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80c7e80 0x7f53a806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 -- 172.21.15.49:0/692627119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80c7e80 msgr2=0x7f53a806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:23.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80c7e80 0x7f53a806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 -- 172.21.15.49:0/692627119 >> v1:172.21.15.49:6789/0 conn(0x7f53a81551d0 legacy=0x7f53a815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a67fc640 1 -- 172.21.15.49:0/692627119 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951551656 0 0) 0x7f53a815c700 con 0x7f53a81551d0 2024-05-09T08:58:23.479 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.478+0000 7f53a77fe640 1 -- 172.21.15.49:0/692627119 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a815f030 con 0x7f53a8154df0 2024-05-09T08:58:23.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.479+0000 7f53a77fe640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a815c1c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5398009870 tx=0x7f539802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28494175a1acb916 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:23.480 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.479+0000 7f53a67fc640 1 -- 172.21.15.49:0/692627119 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398037070 con 0x7f53a8154df0 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.479+0000 7f53a67fc640 1 -- 172.21.15.49:0/692627119 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5398032440 con 0x7f53a8154df0 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.479+0000 7f53a67fc640 1 -- 172.21.15.49:0/692627119 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398032810 con 0x7f53a8154df0 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 -- 172.21.15.49:0/692627119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 msgr2=0x7f53a815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a815c1c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5398009870 tx=0x7f539802a270 comp rx=0 tx=0).stop 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 -- 172.21.15.49:0/692627119 shutdown_connections 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80c7e80 0x7f53a806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/692627119 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a815c1c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 -- 172.21.15.49:0/692627119 >> 172.21.15.49:0/692627119 conn(0x7f53a806b200 msgr2=0x7f53a806b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 -- 172.21.15.49:0/692627119 shutdown_connections 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.480+0000 7f53ae1c1640 1 -- 172.21.15.49:0/692627119 wait complete. 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.481+0000 7f53ae1c1640 1 Processor -- start 2024-05-09T08:58:23.481 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.481+0000 7f53ae1c1640 1 -- start start 2024-05-09T08:58:23.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80c7e80 0x7f53a81f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80cc700 0x7f53a81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f53a815b480 con 0x7f53a8154df0 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53a815b1a0 con 0x7f53a80c7e80 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53ae1c1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f53a815e9d0 con 0x7f53a80cc700 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45280/0 (socket says 172.21.15.49:45280) 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 -- 172.21.15.49:0/1840306301 learned_addr learned my addr 172.21.15.49:0/1840306301 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a7fff640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80cc700 0x7f53a81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a77fe640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80c7e80 0x7f53a81f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 -- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80cc700 msgr2=0x7f53a81571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80cc700 0x7f53a81571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 -- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80c7e80 msgr2=0x7f53a81f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:23.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80c7e80 0x7f53a81f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.482+0000 7f53a6ffd640 1 -- 172.21.15.49:0/1840306301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a81f14b0 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f53a6ffd640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f539c00c3e0 tx=0x7f539c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c0074e0 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539c007680 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a80cb520 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c004a20 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.483+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a81f0770 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.484+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a80cba40 con 0x7f53a8154df0 2024-05-09T08:58:23.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.484+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f539c01a020 con 0x7f53a8154df0 2024-05-09T08:58:23.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.485+0000 7f538ffff640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:23.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.485+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5886+0+0 (secure 0 0 0) 0x7f539c08a660 con 0x7f53a8154df0 2024-05-09T08:58:23.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.485+0000 7f53a77fe640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:23.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.486+0000 7f53a77fe640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5398034000 tx=0x7f539802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:23.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.489+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f539c058880 con 0x7f53a8154df0 2024-05-09T08:58:23.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:23.685+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f53a81f0560 con 0x7f53a8154df0 2024-05-09T08:58:24.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.355+0000 7f538ffff640 1 -- 172.21.15.49:0/1840306301 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v77) ==== 113+0+0 (secure 0 0 0) 0x7f539c05cb80 con 0x7f53a8154df0 2024-05-09T08:58:24.355 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 msgr2=0x7f539406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5398034000 tx=0x7f539802e000 comp rx=0 tx=0).stop 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 msgr2=0x7f53a814b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f539c00c3e0 tx=0x7f539c00c8a0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 shutdown_connections 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f539406be30 0x7f539406e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f53a80cc700 0x7f53a81571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f53a8154df0 0x7f53a814b110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 --2- 172.21.15.49:0/1840306301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a80c7e80 0x7f53a81f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 >> 172.21.15.49:0/1840306301 conn(0x7f53a806b200 msgr2=0x7f53a8071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 shutdown_connections 2024-05-09T08:58:24.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.359+0000 7f53ae1c1640 1 -- 172.21.15.49:0/1840306301 wait complete. 2024-05-09T08:58:24.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-05-09T08:58:24.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.487+0000 7fec78d42640 1 Processor -- start 2024-05-09T08:58:24.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.487+0000 7fec78d42640 1 -- start start 2024-05-09T08:58:24.487 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec78d42640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec74155090 0x7fec740bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec78d42640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fec740c9ec0 0x7fec7415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec78d42640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7406cdc0 con 0x7fec740c71a0 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec78d42640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fec740c8a20 con 0x7fec74155090 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec78d42640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fec7406d080 con 0x7fec740c9ec0 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec7259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fec74155090 0x7fec740bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec71d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fec740c71a0 0x7fec7415e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47524/0 (socket says 172.21.15.49:47524) 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec71d9b640 1 -- 172.21.15.49:0/2911228048 learned_addr learned my addr 172.21.15.49:0/2911228048 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec72d9d640 1 --2- 172.21.15.49:0/2911228048 >> v2:172.21.15.183:3300/0 conn(0x7fec740c9ec0 0x7fec7415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec7259c640 1 -- 172.21.15.49:0/2911228048 >> v2:172.21.15.183:3300/0 conn(0x7fec740c9ec0 msgr2=0x7fec7415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec7259c640 1 --2- 172.21.15.49:0/2911228048 >> v2:172.21.15.183:3300/0 conn(0x7fec740c9ec0 0x7fec7415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec7259c640 1 -- 172.21.15.49:0/2911228048 >> v1:172.21.15.49:6789/0 conn(0x7fec740c71a0 legacy=0x7fec7415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.488+0000 7fec7259c640 1 -- 172.21.15.49:0/2911228048 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7406d360 con 0x7fec74155090 2024-05-09T08:58:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.489+0000 7fec7259c640 1 --2- 172.21.15.49:0/2911228048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec74155090 0x7fec740bf490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec740be470 tx=0x7fec6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e2ebf8e85b67b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.489+0000 7fec7159a640 1 -- 172.21.15.49:0/2911228048 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64037070 con 0x7fec74155090 2024-05-09T08:58:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.489+0000 7fec7159a640 1 -- 172.21.15.49:0/2911228048 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64032440 con 0x7fec74155090 2024-05-09T08:58:24.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.489+0000 7fec7159a640 1 -- 172.21.15.49:0/2911228048 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64032810 con 0x7fec74155090 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 -- 172.21.15.49:0/2911228048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec74155090 msgr2=0x7fec740bf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 --2- 172.21.15.49:0/2911228048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec74155090 0x7fec740bf490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec740be470 tx=0x7fec6402a230 comp rx=0 tx=0).stop 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 -- 172.21.15.49:0/2911228048 shutdown_connections 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 --2- 172.21.15.49:0/2911228048 >> v2:172.21.15.183:3300/0 conn(0x7fec740c9ec0 0x7fec7415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 --2- 172.21.15.49:0/2911228048 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec74155090 0x7fec740bf490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 -- 172.21.15.49:0/2911228048 >> 172.21.15.49:0/2911228048 conn(0x7fec740c93a0 msgr2=0x7fec740c97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 -- 172.21.15.49:0/2911228048 shutdown_connections 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.490+0000 7fec78d42640 1 -- 172.21.15.49:0/2911228048 wait complete. 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.491+0000 7fec78d42640 1 Processor -- start 2024-05-09T08:58:24.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.491+0000 7fec78d42640 1 -- start start 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fec740c9ec0 0x7fec741f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec74155090 0x7fec741f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fec740bec90 con 0x7fec740c71a0 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec740be9b0 con 0x7fec74155090 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec78d42640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fec740bef70 con 0x7fec740c9ec0 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45292/0 (socket says 172.21.15.49:45292) 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 -- 172.21.15.49:0/1462537332 learned_addr learned my addr 172.21.15.49:0/1462537332 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec71d9b640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fec740c9ec0 0x7fec741f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec72d9d640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec74155090 0x7fec741f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 -- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fec740c9ec0 msgr2=0x7fec741f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fec740c9ec0 0x7fec741f1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 -- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec74155090 msgr2=0x7fec741f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec74155090 0x7fec741f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 -- 172.21.15.49:0/1462537332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec740735b0 con 0x7fec740c71a0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.492+0000 7fec7259c640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fec64009890 tx=0x7fec640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.493+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64037040 con 0x7fec740c71a0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.493+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64042070 con 0x7fec740c71a0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.493+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec74154290 con 0x7fec740c71a0 2024-05-09T08:58:24.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.493+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64030bd0 con 0x7fec740c71a0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.493+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7415b880 con 0x7fec740c71a0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.494+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7404f660 con 0x7fec740c71a0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.494+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec64047460 con 0x7fec740c71a0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.495+0000 7fec62ffd640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.495+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5886+0+0 (secure 0 0 0) 0x7fec640abae0 con 0x7fec740c71a0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.495+0000 7fec71d9b640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:24.497 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.495+0000 7fec71d9b640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 0x7fec5406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec74073940 tx=0x7fec68009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:24.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.499+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fec64032e20 con 0x7fec740c71a0 2024-05-09T08:58:24.694 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:24.695+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7fec741587d0 con 0x7fec740c71a0 2024-05-09T08:58:25.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.357+0000 7fec62ffd640 1 -- 172.21.15.49:0/1462537332 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v78) ==== 133+0+0 (secure 0 0 0) 0x7fec6407a3e0 con 0x7fec740c71a0 2024-05-09T08:58:25.356 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-05-09T08:58:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 msgr2=0x7fec5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 0x7fec5406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec74073940 tx=0x7fec68009040 comp rx=0 tx=0).stop 2024-05-09T08:58:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 msgr2=0x7fec74151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fec64009890 tx=0x7fec640046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 shutdown_connections 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec74155090 0x7fec741f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fec740c9ec0 0x7fec741f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 --2- 172.21.15.49:0/1462537332 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fec740c71a0 0x7fec74151100 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 >> 172.21.15.49:0/1462537332 conn(0x7fec740c93a0 msgr2=0x7fec7414fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 shutdown_connections 2024-05-09T08:58:25.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.360+0000 7fec78d42640 1 -- 172.21.15.49:0/1462537332 wait complete. 2024-05-09T08:58:25.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-09T08:58:25.483 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.484+0000 7ff9b0237640 1 Processor -- start 2024-05-09T08:58:25.484 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.484+0000 7ff9b0237640 1 -- start start 2024-05-09T08:58:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9b0237640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 0x7ff9a8166610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9b0237640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a815f160 con 0x7ff9a80c8f90 2024-05-09T08:58:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9b0237640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a80c78b0 con 0x7ff9a8155180 2024-05-09T08:58:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9b0237640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a80c8de0 con 0x7ff9a80c8950 2024-05-09T08:58:25.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9ad7d2640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff9a80c8f90 0x7ff9a80c2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47532/0 (socket says 172.21.15.49:47532) 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.485+0000 7ff9ad7d2640 1 -- 172.21.15.49:0/3151850338 learned_addr learned my addr 172.21.15.49:0/3151850338 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9ae7d4640 1 --2- 172.21.15.49:0/3151850338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 0x7ff9a8166610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2060234464 0 0) 0x7ff9a815f160 con 0x7ff9a80c8f90 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff990003520 con 0x7ff9a80c8f90 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058911684 0 0) 0x7ff9a80c78b0 con 0x7ff9a8155180 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9a815f160 con 0x7ff9a8155180 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9ae7d4640 1 -- 172.21.15.49:0/3151850338 >> v1:172.21.15.183:6789/0 conn(0x7ff9a8155180 legacy=0x7ff9a806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9ae7d4640 1 -- 172.21.15.49:0/3151850338 >> v1:172.21.15.49:6789/0 conn(0x7ff9a80c8f90 legacy=0x7ff9a80c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.486 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.486+0000 7ff9ae7d4640 1 -- 172.21.15.49:0/3151850338 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a815e3a0 con 0x7ff9a80c8950 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9ae7d4640 1 --2- 172.21.15.49:0/3151850338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 0x7ff9a8166610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097d0 tx=0x7ff9a002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a09714afbb4cf19 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0037070 con 0x7ff9a80c8950 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a0032440 con 0x7ff9a80c8950 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9acfd1640 1 -- 172.21.15.49:0/3151850338 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00327f0 con 0x7ff9a80c8950 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9b0237640 1 -- 172.21.15.49:0/3151850338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 msgr2=0x7ff9a8166610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9b0237640 1 --2- 172.21.15.49:0/3151850338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 0x7ff9a8166610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00097d0 tx=0x7ff9a002a180 comp rx=0 tx=0).stop 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9b0237640 1 -- 172.21.15.49:0/3151850338 shutdown_connections 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9b0237640 1 --2- 172.21.15.49:0/3151850338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a80c8950 0x7ff9a8166610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.487+0000 7ff9b0237640 1 -- 172.21.15.49:0/3151850338 >> 172.21.15.49:0/3151850338 conn(0x7ff9a80c9650 msgr2=0x7ff9a80c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.488+0000 7ff9b0237640 1 -- 172.21.15.49:0/3151850338 shutdown_connections 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.488+0000 7ff9b0237640 1 -- 172.21.15.49:0/3151850338 wait complete. 2024-05-09T08:58:25.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.489+0000 7ff9b0237640 1 Processor -- start 2024-05-09T08:58:25.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.489+0000 7ff9b0237640 1 -- start start 2024-05-09T08:58:25.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.489+0000 7ff9b0237640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.489+0000 7ff9b0237640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a80c8f90 0x7ff9a806bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a8155180 0x7ff9a81f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a815e960 con 0x7ff9a80c8950 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a80c7b90 con 0x7ff9a80c8f90 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a806b910 con 0x7ff9a8155180 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45300/0 (socket says 172.21.15.49:45300) 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 -- 172.21.15.49:0/1859242230 learned_addr learned my addr 172.21.15.49:0/1859242230 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9ae7d4640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a8155180 0x7ff9a81f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9ad7d2640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a80c8f90 0x7ff9a806bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 -- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a8155180 msgr2=0x7ff9a81f0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a8155180 0x7ff9a81f0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 -- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a80c8f90 msgr2=0x7ff9a806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a80c8f90 0x7ff9a806bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 -- 172.21.15.49:0/1859242230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a81f49b0 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9adfd3640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c7a0 tx=0x7ff99800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998003b70 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998003d10 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a815dbb0 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99800aa90 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.490+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a81f09b0 con 0x7ff9a80c8950 2024-05-09T08:58:25.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.491+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a806ddf0 con 0x7ff9a80c8950 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.492+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff99800ac30 con 0x7ff9a80c8950 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.492+0000 7ff9a67fc640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 0x7ff99406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.492+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7ff99808acc0 con 0x7ff9a80c8950 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.493+0000 7ff9ad7d2640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 0x7ff99406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:25.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.493+0000 7ff9ad7d2640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 0x7ff99406e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0011e0 tx=0x7ff99c007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:25.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.497+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff9a806ddf0 con 0x7ff9a80c8950 2024-05-09T08:58:25.690 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:25.691+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7ff9a804f660 con 0x7ff9a80c8950 2024-05-09T08:58:26.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.363+0000 7ff9a67fc640 1 -- 172.21.15.49:0/1859242230 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v79) ==== 113+0+0 (secure 0 0 0) 0x7ff998059000 con 0x7ff9a80c8950 2024-05-09T08:58:26.363 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-05-09T08:58:26.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.366+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 msgr2=0x7ff99406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.366+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 0x7ff99406e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0011e0 tx=0x7ff99c007d30 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.366+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 msgr2=0x7ff9a80c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.366+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff99800c7a0 tx=0x7ff99800cc60 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 shutdown_connections 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff99406be60 0x7ff99406e320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9a8155180 0x7ff9a81f0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9a80c8f90 0x7ff9a806bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 --2- 172.21.15.49:0/1859242230 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff9a80c8950 0x7ff9a80c4c60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 >> 172.21.15.49:0/1859242230 conn(0x7ff9a80c9650 msgr2=0x7ff9a8152860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 shutdown_connections 2024-05-09T08:58:26.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.367+0000 7ff9b0237640 1 -- 172.21.15.49:0/1859242230 wait complete. 2024-05-09T08:58:26.380 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-05-09T08:58:26.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.491+0000 7f03aaa2a640 1 Processor -- start 2024-05-09T08:58:26.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.491+0000 7f03aaa2a640 1 -- start start 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03a4152e30 0x7f03a4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f03a40c7130 0x7f03a40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 0x7f03a40c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f03a4159b10 con 0x7f03a4152e30 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f03a4159270 con 0x7f03a40c7130 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03aaa2a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f03a4159550 con 0x7f03a40c6450 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03a4152e30 0x7f03a4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f03a4152e30 0x7f03a4158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45312/0 (socket says 172.21.15.49:45312) 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.492+0000 7f03a3fff640 1 -- 172.21.15.49:0/3777271464 learned_addr learned my addr 172.21.15.49:0/3777271464 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a37fe640 1 --2- 172.21.15.49:0/3777271464 >> v2:172.21.15.183:3300/0 conn(0x7f03a40c7130 0x7f03a40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a8fc7640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 0x7f03a40c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 -- 172.21.15.49:0/3777271464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 msgr2=0x7f03a40c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 0x7f03a40c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 -- 172.21.15.49:0/3777271464 >> v2:172.21.15.183:3300/0 conn(0x7f03a40c7130 msgr2=0x7f03a40c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 --2- 172.21.15.49:0/3777271464 >> v2:172.21.15.183:3300/0 conn(0x7f03a40c7130 0x7f03a40c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 -- 172.21.15.49:0/3777271464 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4159830 con 0x7f03a4152e30 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a8fc7640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 0x7f03a40c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.493+0000 7f03a3fff640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a4152e30 0x7f03a4158be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03a406d1a0 tx=0x7f039402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c5ecde2b6ed5b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03a2ffd640 1 -- 172.21.15.49:0/3777271464 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394037070 con 0x7f03a4152e30 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03a2ffd640 1 -- 172.21.15.49:0/3777271464 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0394032440 con 0x7f03a4152e30 2024-05-09T08:58:26.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03a2ffd640 1 -- 172.21.15.49:0/3777271464 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394032810 con 0x7f03a4152e30 2024-05-09T08:58:26.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 -- 172.21.15.49:0/3777271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a4152e30 msgr2=0x7f03a4158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:26.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a4152e30 0x7f03a4158be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03a406d1a0 tx=0x7f039402a1e0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 -- 172.21.15.49:0/3777271464 shutdown_connections 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a40c6450 0x7f03a40c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/3777271464 >> v2:172.21.15.183:3300/0 conn(0x7f03a40c7130 0x7f03a40c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/3777271464 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a4152e30 0x7f03a4158be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 -- 172.21.15.49:0/3777271464 >> 172.21.15.49:0/3777271464 conn(0x7f03a40cc290 msgr2=0x7f03a40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 -- 172.21.15.49:0/3777271464 shutdown_connections 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.494+0000 7f03aaa2a640 1 -- 172.21.15.49:0/3777271464 wait complete. 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.495+0000 7f03aaa2a640 1 Processor -- start 2024-05-09T08:58:26.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 -- start start 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 0x7f03a414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 0x7f03a41f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f03a406b4b0 con 0x7f03a40c6450 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03a40c59f0 con 0x7f03a40c7130 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03aaa2a640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f03a4153210 con 0x7f03a4152e30 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45328/0 (socket says 172.21.15.49:45328) 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.496+0000 7f03a3fff640 1 -- 172.21.15.49:0/659296566 learned_addr learned my addr 172.21.15.49:0/659296566 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a37fe640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 0x7f03a414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a8fc7640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 0x7f03a41f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 -- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 msgr2=0x7f03a41f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 0x7f03a41f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 -- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 msgr2=0x7f03a414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 0x7f03a414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 -- 172.21.15.49:0/659296566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a40c8fa0 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a8fc7640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 0x7f03a41f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a37fe640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 0x7f03a414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a3fff640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f039402a220 tx=0x7f03940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394041020 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03940306c0 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394037040 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a40c9280 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.497+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a40c9840 con 0x7f03a40c6450 2024-05-09T08:58:26.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.498+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a404f660 con 0x7f03a40c6450 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.499+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0394044020 con 0x7f03a40c6450 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.499+0000 7f03a0ff9640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 0x7f039006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.499+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5886+0+0 (secure 0 0 0) 0x7f03940ab750 con 0x7f03a40c6450 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.499+0000 7f03a37fe640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 0x7f039006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.500+0000 7f03a37fe640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 0x7f039006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03a4149470 tx=0x7f0398008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:26.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.503+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f039404b030 con 0x7f03a40c6450 2024-05-09T08:58:26.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:26.699+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7f03a40c3470 con 0x7f03a40c6450 2024-05-09T08:58:27.371 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.371+0000 7f03a0ff9640 1 -- 172.21.15.49:0/659296566 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v80) ==== 133+0+0 (secure 0 0 0) 0x7f039407a050 con 0x7f03a40c6450 2024-05-09T08:58:27.371 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-05-09T08:58:27.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.374+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 msgr2=0x7f039006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.373 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.374+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 0x7f039006e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03a4149470 tx=0x7f0398008040 comp rx=0 tx=0).stop 2024-05-09T08:58:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.374+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 msgr2=0x7f03a4158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.374+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f039402a220 tx=0x7f03940046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 shutdown_connections 2024-05-09T08:58:27.374 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f039006be30 0x7f039006e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03a4152e30 0x7f03a41f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03a40c7130 0x7f03a414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 --2- 172.21.15.49:0/659296566 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f03a40c6450 0x7f03a4158850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 >> 172.21.15.49:0/659296566 conn(0x7f03a40cc290 msgr2=0x7f03a40c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 shutdown_connections 2024-05-09T08:58:27.375 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.375+0000 7f03aaa2a640 1 -- 172.21.15.49:0/659296566 wait complete. 2024-05-09T08:58:27.389 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-09T08:58:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.502+0000 7f8baf129640 1 Processor -- start 2024-05-09T08:58:27.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.502+0000 7f8baf129640 1 -- start start 2024-05-09T08:58:27.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8baf129640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ba8152e30 0x7f8ba8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:27.502 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8baf129640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba8159b10 con 0x7f8ba80c7130 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8baf129640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba8159270 con 0x7f8ba8072670 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8baf129640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba8159550 con 0x7f8ba8152e30 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8bacec5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ba8152e30 0x7f8ba8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8ba7fff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8ba80c7130 0x7f8ba80c5f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47548/0 (socket says 172.21.15.49:47548) 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8bacec5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f8ba8152e30 0x7f8ba8158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45344/0 (socket says 172.21.15.49:45344) 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8ba7fff640 1 -- 172.21.15.49:0/3047271912 learned_addr learned my addr 172.21.15.49:0/3047271912 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8bacec5640 1 -- 172.21.15.49:0/3047271912 >> v1:172.21.15.183:6789/0 conn(0x7f8ba8072670 legacy=0x7f8ba80c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8bacec5640 1 -- 172.21.15.49:0/3047271912 >> v1:172.21.15.49:6789/0 conn(0x7f8ba80c7130 legacy=0x7f8ba80c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8bacec5640 1 -- 172.21.15.49:0/3047271912 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8159830 con 0x7f8ba8152e30 2024-05-09T08:58:27.503 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.503+0000 7f8ba77fe640 1 -- 172.21.15.49:0/3047271912 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270262068 0 0) 0x7f8ba8159b10 con 0x7f8ba80c7130 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8bacec5640 1 --2- 172.21.15.49:0/3047271912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba8152e30 0x7f8ba8158be0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0097a0 tx=0x7f8b9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8d917ba85f423b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8ba77fe640 1 -- 172.21.15.49:0/3047271912 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c037070 con 0x7f8ba8152e30 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8ba77fe640 1 -- 172.21.15.49:0/3047271912 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c032440 con 0x7f8ba8152e30 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8ba77fe640 1 -- 172.21.15.49:0/3047271912 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c032810 con 0x7f8ba8152e30 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8baf129640 1 -- 172.21.15.49:0/3047271912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba8152e30 msgr2=0x7f8ba8158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.504+0000 7f8baf129640 1 --2- 172.21.15.49:0/3047271912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba8152e30 0x7f8ba8158be0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0097a0 tx=0x7f8b9c02a1e0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.505+0000 7f8baf129640 1 -- 172.21.15.49:0/3047271912 shutdown_connections 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.505+0000 7f8baf129640 1 --2- 172.21.15.49:0/3047271912 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba8152e30 0x7f8ba8158be0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.505+0000 7f8baf129640 1 -- 172.21.15.49:0/3047271912 >> 172.21.15.49:0/3047271912 conn(0x7f8ba80cc290 msgr2=0x7f8ba80cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.505+0000 7f8baf129640 1 -- 172.21.15.49:0/3047271912 shutdown_connections 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.505+0000 7f8baf129640 1 -- 172.21.15.49:0/3047271912 wait complete. 2024-05-09T08:58:27.505 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.506+0000 7f8baf129640 1 Processor -- start 2024-05-09T08:58:27.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.506+0000 7f8baf129640 1 -- start start 2024-05-09T08:58:27.507 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba8072670 0x7f8ba814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba8152e30 0x7f8ba80caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80c6620 con 0x7f8ba80c7130 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80c59f0 con 0x7f8ba8072670 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8baf129640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba806b4b0 con 0x7f8ba8152e30 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45350/0 (socket says 172.21.15.49:45350) 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 -- 172.21.15.49:0/2945111253 learned_addr learned my addr 172.21.15.49:0/2945111253 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8bacec5640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba8072670 0x7f8ba814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8bad6c6640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba8152e30 0x7f8ba80caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 -- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba8152e30 msgr2=0x7f8ba80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba8152e30 0x7f8ba80caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 -- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba8072670 msgr2=0x7f8ba814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba8072670 0x7f8ba814ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 -- 172.21.15.49:0/2945111253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba814e930 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.507+0000 7f8ba7fff640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400c500 tx=0x7f8b9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.508+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94018070 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.508+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b94007580 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.508+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94004a20 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.508+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba80cb9b0 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.508+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81ef280 con 0x7f8ba80c7130 2024-05-09T08:58:27.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.509+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba804f660 con 0x7f8ba80c7130 2024-05-09T08:58:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.509+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b9401b070 con 0x7f8ba80c7130 2024-05-09T08:58:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.510+0000 7f8ba57fa640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 0x7f8b8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.510+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5886+0+0 (secure 0 0 0) 0x7f8b9408a0e0 con 0x7f8ba80c7130 2024-05-09T08:58:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.510+0000 7f8bacec5640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 0x7f8b8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:27.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.511+0000 7f8bacec5640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 0x7f8b8c06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c034000 tx=0x7f8b9c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:27.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.514+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ba80cb9b0 con 0x7f8ba80c7130 2024-05-09T08:58:27.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:27.709+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f8ba80c3470 con 0x7f8ba80c7130 2024-05-09T08:58:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.380+0000 7f8ba57fa640 1 -- 172.21.15.49:0/2945111253 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v81) ==== 113+0+0 (secure 0 0 0) 0x7f8b94058420 con 0x7f8ba80c7130 2024-05-09T08:58:28.379 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 msgr2=0x7f8b8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 0x7f8b8c06e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c034000 tx=0x7f8b9c02e000 comp rx=0 tx=0).stop 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 msgr2=0x7f8ba814d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400c500 tx=0x7f8b9400c9c0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 shutdown_connections 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8b8c06be30 0x7f8b8c06e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba8152e30 0x7f8ba80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.383 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8ba80c7130 0x7f8ba814d9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 --2- 172.21.15.49:0/2945111253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba8072670 0x7f8ba814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 >> 172.21.15.49:0/2945111253 conn(0x7f8ba80cc290 msgr2=0x7f8ba81587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 shutdown_connections 2024-05-09T08:58:28.384 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.383+0000 7f8baf129640 1 -- 172.21.15.49:0/2945111253 wait complete. 2024-05-09T08:58:28.396 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-09T08:58:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.507+0000 7f1c7977e640 1 Processor -- start 2024-05-09T08:58:28.506 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.507+0000 7f1c7977e640 1 -- start start 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c7977e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c740c82b0 0x7f1c74159550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c7977e640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7406ed80 con 0x7f1c740c6160 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c7977e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7406f040 con 0x7f1c74152aa0 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c7977e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1c7406f320 con 0x7f1c740c82b0 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c737fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1c740c82b0 0x7f1c74159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:28.508 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c727fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f1c740c6160 0x7f1c7415c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47564/0 (socket says 172.21.15.49:47564) 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c727fc640 1 -- 172.21.15.49:0/2899147911 learned_addr learned my addr 172.21.15.49:0/2899147911 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c737fe640 1 -- 172.21.15.49:0/2899147911 >> v1:172.21.15.183:6789/0 conn(0x7f1c74152aa0 legacy=0x7f1c740c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c737fe640 1 -- 172.21.15.49:0/2899147911 >> v1:172.21.15.49:6789/0 conn(0x7f1c740c6160 legacy=0x7f1c7415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c71ffb640 1 -- 172.21.15.49:0/2899147911 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935645563 0 0) 0x7f1c7406ed80 con 0x7f1c740c6160 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.508+0000 7f1c737fe640 1 -- 172.21.15.49:0/2899147911 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c740c4b90 con 0x7f1c740c82b0 2024-05-09T08:58:28.509 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c737fe640 1 --2- 172.21.15.49:0/2899147911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c740c82b0 0x7f1c74159550 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009d90 tx=0x7f1c6c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dff5e9c197a409 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c71ffb640 1 -- 172.21.15.49:0/2899147911 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c007740 con 0x7f1c740c82b0 2024-05-09T08:58:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c71ffb640 1 -- 172.21.15.49:0/2899147911 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c0078e0 con 0x7f1c740c82b0 2024-05-09T08:58:28.510 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c71ffb640 1 -- 172.21.15.49:0/2899147911 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c007cb0 con 0x7f1c740c82b0 2024-05-09T08:58:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c7977e640 1 -- 172.21.15.49:0/2899147911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c740c82b0 msgr2=0x7f1c74159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.509+0000 7f1c7977e640 1 --2- 172.21.15.49:0/2899147911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c740c82b0 0x7f1c74159550 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009d90 tx=0x7f1c6c02a730 comp rx=0 tx=0).stop 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.510+0000 7f1c7977e640 1 -- 172.21.15.49:0/2899147911 shutdown_connections 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.510+0000 7f1c7977e640 1 --2- 172.21.15.49:0/2899147911 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c740c82b0 0x7f1c74159550 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.510+0000 7f1c7977e640 1 -- 172.21.15.49:0/2899147911 >> 172.21.15.49:0/2899147911 conn(0x7f1c7406b200 msgr2=0x7f1c7406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.510+0000 7f1c7977e640 1 -- 172.21.15.49:0/2899147911 shutdown_connections 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.510+0000 7f1c7977e640 1 -- 172.21.15.49:0/2899147911 wait complete. 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.511+0000 7f1c7977e640 1 Processor -- start 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.511+0000 7f1c7977e640 1 -- start start 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c740c6160 0x7f1c740cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 0x7f1c741e2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1c74152e80 con 0x7f1c74152aa0 2024-05-09T08:58:28.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c740c5150 con 0x7f1c740c6160 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1c741f4ed0 con 0x7f1c740c82b0 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45360/0 (socket says 172.21.15.49:45360) 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 -- 172.21.15.49:0/1424556438 learned_addr learned my addr 172.21.15.49:0/1424556438 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c72ffd640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c740c6160 0x7f1c740cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c727fc640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 0x7f1c741e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 -- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 msgr2=0x7f1c741e2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 0x7f1c741e2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 -- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c740c6160 msgr2=0x7f1c740cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c740c6160 0x7f1c740cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 -- 172.21.15.49:0/1424556438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c740730d0 con 0x7f1c74152aa0 2024-05-09T08:58:28.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c727fc640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 0x7f1c741e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c737fe640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c02ad20 tx=0x7f1c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c040070 con 0x7f1c74152aa0 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6c031c90 con 0x7f1c74152aa0 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c741f42c0 con 0x7f1c74152aa0 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c045680 con 0x7f1c74152aa0 2024-05-09T08:58:28.516 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.512+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c741f4880 con 0x7f1c74152aa0 2024-05-09T08:58:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.514+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c7404f660 con 0x7f1c74152aa0 2024-05-09T08:58:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.514+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c6c039040 con 0x7f1c74152aa0 2024-05-09T08:58:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.515+0000 7f1c5b7fe640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:28.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.515+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5886+0+0 (secure 0 0 0) 0x7f1c6c0ac920 con 0x7f1c74152aa0 2024-05-09T08:58:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.516+0000 7f1c72ffd640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:28.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.516+0000 7f1c72ffd640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 0x7f1c6006e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c64006fd0 tx=0x7f1c64008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:28.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.520+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c6c043360 con 0x7f1c74152aa0 2024-05-09T08:58:28.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:28.715+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7f1c74068c70 con 0x7f1c74152aa0 2024-05-09T08:58:29.388 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.389+0000 7f1c5b7fe640 1 -- 172.21.15.49:0/1424556438 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v82) ==== 133+0+0 (secure 0 0 0) 0x7f1c6c081070 con 0x7f1c74152aa0 2024-05-09T08:58:29.388 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-09T08:58:29.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 msgr2=0x7f1c6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 0x7f1c6006e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c64006fd0 tx=0x7f1c64008040 comp rx=0 tx=0).stop 2024-05-09T08:58:29.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 msgr2=0x7f1c74072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c02ad20 tx=0x7f1c6c004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 shutdown_connections 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1c6006be30 0x7f1c6006e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1c74152aa0 0x7f1c74072620 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c740c82b0 0x7f1c741e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 --2- 172.21.15.49:0/1424556438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c740c6160 0x7f1c740cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 >> 172.21.15.49:0/1424556438 conn(0x7f1c7406b200 msgr2=0x7f1c740c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 shutdown_connections 2024-05-09T08:58:29.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.392+0000 7f1c7977e640 1 -- 172.21.15.49:0/1424556438 wait complete. 2024-05-09T08:58:29.404 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-09T08:58:29.404 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-09T08:58:29.404 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-09T08:58:29.405 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-09T08:58:29.405 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T08:58:29.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:58:29.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.513+0000 7f5ca405d640 1 Processor -- start 2024-05-09T08:58:29.513 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 -- start start 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c150ed0 0x7f5c9c0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c0c7ea0 con 0x7f5c9c0700f0 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c06d7a0 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.514+0000 7f5ca405d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c06da80 con 0x7f5c9c150ed0 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.514 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca1df9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5c9c0700f0 0x7f5c9c06e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47578/0 (socket says 172.21.15.49:47578) 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca1df9640 1 -- 172.21.15.49:0/2346709480 learned_addr learned my addr 172.21.15.49:0/2346709480 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 --2- 172.21.15.49:0/2346709480 >> v2:172.21.15.49:3300/0 conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45374/0 (socket says 172.21.15.49:45374) 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca25fa640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c150ed0 0x7f5c9c0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 -- 172.21.15.49:0/2346709480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c150ed0 msgr2=0x7f5c9c0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c150ed0 0x7f5c9c0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 -- 172.21.15.49:0/2346709480 >> v1:172.21.15.49:6789/0 conn(0x7f5c9c0700f0 legacy=0x7f5c9c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca0df7640 1 -- 172.21.15.49:0/2346709480 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818426255 0 0) 0x7f5c9c0c7ea0 con 0x7f5c9c0700f0 2024-05-09T08:58:29.515 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.515+0000 7f5ca15f8640 1 -- 172.21.15.49:0/2346709480 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c06dd60 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca15f8640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097a0 tx=0x7f5c9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d9036cc9b4439 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca0df7640 1 -- 172.21.15.49:0/2346709480 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90037070 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca0df7640 1 -- 172.21.15.49:0/2346709480 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c90032440 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca0df7640 1 -- 172.21.15.49:0/2346709480 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90032810 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 -- 172.21.15.49:0/2346709480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c0c8a10 msgr2=0x7f5c9c0c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5c900097a0 tx=0x7f5c9002a210 comp rx=0 tx=0).stop 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 -- 172.21.15.49:0/2346709480 shutdown_connections 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c150ed0 0x7f5c9c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 --2- 172.21.15.49:0/2346709480 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c0c8a10 0x7f5c9c0c7810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.516+0000 7f5ca405d640 1 -- 172.21.15.49:0/2346709480 >> 172.21.15.49:0/2346709480 conn(0x7f5c9c074840 msgr2=0x7f5c9c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:29.517 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.517+0000 7f5ca405d640 1 -- 172.21.15.49:0/2346709480 shutdown_connections 2024-05-09T08:58:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.517+0000 7f5ca405d640 1 -- 172.21.15.49:0/2346709480 wait complete. 2024-05-09T08:58:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.518+0000 7f5ca405d640 1 Processor -- start 2024-05-09T08:58:29.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.518+0000 7f5ca405d640 1 -- start start 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.518+0000 7f5ca405d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9c0700f0 0x7f5c9c153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.518+0000 7f5ca405d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c0c8a10 0x7f5c9c1e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c9440 con 0x7f5c9c150ed0 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c151360 con 0x7f5c9c0700f0 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c0c95c0 con 0x7f5c9c0c8a10 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45380/0 (socket says 172.21.15.49:45380) 2024-05-09T08:58:29.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 -- 172.21.15.49:0/381147569 learned_addr learned my addr 172.21.15.49:0/381147569 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca15f8640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c0c8a10 0x7f5c9c1e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca1df9640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9c0700f0 0x7f5c9c153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 -- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c0c8a10 msgr2=0x7f5c9c1e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c0c8a10 0x7f5c9c1e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 -- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9c0700f0 msgr2=0x7f5c9c153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9c0700f0 0x7f5c9c153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 -- 172.21.15.49:0/381147569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c154260 con 0x7f5c9c150ed0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca25fa640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007e60 tx=0x7f5c9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94014420 con 0x7f5c9c150ed0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c94019070 con 0x7f5c9c150ed0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c0c4b00 con 0x7f5c9c150ed0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94014820 con 0x7f5c9c150ed0 2024-05-09T08:58:29.521 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.519+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c0c50c0 con 0x7f5c9c150ed0 2024-05-09T08:58:29.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.520+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c068560 con 0x7f5c9c150ed0 2024-05-09T08:58:29.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.521+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c940149c0 con 0x7f5c9c150ed0 2024-05-09T08:58:29.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.521+0000 7f5c8e7fc640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 0x7f5c8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.522 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.521+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f5c9408b810 con 0x7f5c9c150ed0 2024-05-09T08:58:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.522+0000 7f5ca1df9640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 0x7f5c8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.523 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.522+0000 7f5ca1df9640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 0x7f5c8006e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000f60 tx=0x7f5c98007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.526+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c9c154260 con 0x7f5c9c150ed0 2024-05-09T08:58:29.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.714+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c9c0c6e70 con 0x7f5c9c150ed0 2024-05-09T08:58:29.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.715+0000 7f5c8e7fc640 1 -- 172.21.15.49:0/381147569 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16838 (secure 0 0 0) 0x7f5c9405a080 con 0x7f5c9c150ed0 2024-05-09T08:58:29.714 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:29.715 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:58:29.385506+0000","last_up_change":"2024-05-09T08:58:12.270880+0000","last_in_change":"2024-05-09T08:58:13.272774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":38,"up_from":65,"up_thru":66,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6809","nonce":282207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6811","nonce":282207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6815","nonce":282207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6813","nonce":282207482}]},"public_addr":"172.21.15.49:6809/282207482","cluster_addr":"172.21.15.49:6811/282207482","heartbeat_back_addr":"172.21.15.49:6815/282207482","heartbeat_front_addr":"172.21.15.49:6813/282207482","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":39},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":6}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 msgr2=0x7f5c8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 0x7f5c8006e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5c98000f60 tx=0x7f5c98007c00 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 msgr2=0x7f5c9c1e23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007e60 tx=0x7f5c9400e4b0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 shutdown_connections 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c8006be30 0x7f5c8006e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c9c150ed0 0x7f5c9c1e23f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c9c0c8a10 0x7f5c9c1e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 --2- 172.21.15.49:0/381147569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9c0700f0 0x7f5c9c153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 >> 172.21.15.49:0/381147569 conn(0x7f5c9c074840 msgr2=0x7f5c9c154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:29.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 shutdown_connections 2024-05-09T08:58:29.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.717+0000 7f5ca405d640 1 -- 172.21.15.49:0/381147569 wait complete. 2024-05-09T08:58:29.728 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T08:58:29.728 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:58:29.832 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.833+0000 7f41d64a5640 1 Processor -- start 2024-05-09T08:58:29.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.833+0000 7f41d64a5640 1 -- start start 2024-05-09T08:58:29.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41d64a5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41d0153040 0x7f41d00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41d64a5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 0x7f41d00c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41d64a5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d00ca590 con 0x7f41d00c8f70 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41d64a5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f41d00c9980 con 0x7f41d0153040 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41d64a5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f41d00c9c60 con 0x7f41d00cea00 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41cffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41d0153040 0x7f41d00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41cffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f41d0153040 0x7f41d00ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45390/0 (socket says 172.21.15.49:45390) 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.834+0000 7f41cffff640 1 -- 172.21.15.49:0/1675569452 learned_addr learned my addr 172.21.15.49:0/1675569452 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cf7fe640 1 --2- 172.21.15.49:0/1675569452 >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 0x7f41d00c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cffff640 1 -- 172.21.15.49:0/1675569452 >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 msgr2=0x7f41d00c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cffff640 1 --2- 172.21.15.49:0/1675569452 >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 0x7f41d00c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.835 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cffff640 1 -- 172.21.15.49:0/1675569452 >> v1:172.21.15.183:6789/0 conn(0x7f41d00c8f70 legacy=0x7f41d0158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cffff640 1 -- 172.21.15.49:0/1675569452 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d00c9f40 con 0x7f41d0153040 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cf7fe640 1 --2- 172.21.15.49:0/1675569452 >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 0x7f41d00c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41cffff640 1 --2- 172.21.15.49:0/1675569452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d0153040 0x7f41d00ce4c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41d0153420 tx=0x7f41c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49c7a3247c5f08d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.835+0000 7f41ceffd640 1 -- 172.21.15.49:0/1675569452 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037070 con 0x7f41d0153040 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41ceffd640 1 -- 172.21.15.49:0/1675569452 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0032440 con 0x7f41d0153040 2024-05-09T08:58:29.836 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41ceffd640 1 -- 172.21.15.49:0/1675569452 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0032810 con 0x7f41d0153040 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 -- 172.21.15.49:0/1675569452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d0153040 msgr2=0x7f41d00ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 --2- 172.21.15.49:0/1675569452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d0153040 0x7f41d00ce4c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41d0153420 tx=0x7f41c002a230 comp rx=0 tx=0).stop 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 -- 172.21.15.49:0/1675569452 shutdown_connections 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 --2- 172.21.15.49:0/1675569452 >> v2:172.21.15.183:3300/0 conn(0x7f41d00cea00 0x7f41d00c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 --2- 172.21.15.49:0/1675569452 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d0153040 0x7f41d00ce4c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 -- 172.21.15.49:0/1675569452 >> 172.21.15.49:0/1675569452 conn(0x7f41d0067d90 msgr2=0x7f41d0074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 -- 172.21.15.49:0/1675569452 shutdown_connections 2024-05-09T08:58:29.841 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.836+0000 7f41d64a5640 1 -- 172.21.15.49:0/1675569452 wait complete. 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.837+0000 7f41d64a5640 1 Processor -- start 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 -- start start 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f41d00cea00 0x7f41d006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 0x7f41d0148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f41d01596c0 con 0x7f41d00c8f70 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41d0159540 con 0x7f41d0153040 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41d64a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f41d014ac70 con 0x7f41d00cea00 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41cffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45394/0 (socket says 172.21.15.49:45394) 2024-05-09T08:58:29.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.838+0000 7f41cffff640 1 -- 172.21.15.49:0/224170646 learned_addr learned my addr 172.21.15.49:0/224170646 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cf7fe640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f41d00cea00 0x7f41d006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41d4a42640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 0x7f41d0148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 -- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f41d00cea00 msgr2=0x7f41d006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f41d00cea00 0x7f41d006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 -- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 msgr2=0x7f41d0148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 0x7f41d0148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 -- 172.21.15.49:0/224170646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d015a4d0 con 0x7f41d00c8f70 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41d4a42640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 0x7f41d0148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41cffff640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0037040 con 0x7f41d00c8f70 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0042070 con 0x7f41d00c8f70 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d006b2a0 con 0x7f41d00c8f70 2024-05-09T08:58:29.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0030bb0 con 0x7f41d00c8f70 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.839+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d006b860 con 0x7f41d00c8f70 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.840+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d0075850 con 0x7f41d00c8f70 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.841+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41c0047460 con 0x7f41d00c8f70 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.841+0000 7f41ccff9640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.841+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f41c00abb50 con 0x7f41d00c8f70 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.841+0000 7f41cf7fe640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:29.844 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.842+0000 7f41cf7fe640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 0x7f41bc06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41d015a1f0 tx=0x7f41c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:29.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:29.845+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41d0075850 con 0x7f41d00c8f70 2024-05-09T08:58:30.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.036+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41d00ccaf0 con 0x7f41d00c8f70 2024-05-09T08:58:30.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.037+0000 7f41ccff9640 1 -- 172.21.15.49:0/224170646 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16838 (secure 0 0 0) 0x7f41d00ccaf0 con 0x7f41d00c8f70 2024-05-09T08:58:30.037 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:30.037 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":82,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:58:29.385506+0000","last_up_change":"2024-05-09T08:58:12.270880+0000","last_in_change":"2024-05-09T08:58:13.272774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":38,"up_from":65,"up_thru":66,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6809","nonce":282207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6811","nonce":282207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6815","nonce":282207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6813","nonce":282207482}]},"public_addr":"172.21.15.49:6809/282207482","cluster_addr":"172.21.15.49:6811/282207482","heartbeat_back_addr":"172.21.15.49:6815/282207482","heartbeat_front_addr":"172.21.15.49:6813/282207482","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":39},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":6}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 msgr2=0x7f41bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 0x7f41bc06e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41d015a1f0 tx=0x7f41c4008040 comp rx=0 tx=0).stop 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 msgr2=0x7f41d01517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c00046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 shutdown_connections 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f41bc06be30 0x7f41bc06e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.039 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41d0153040 0x7f41d0148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f41d00cea00 0x7f41d006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 --2- 172.21.15.49:0/224170646 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f41d00c8f70 0x7f41d01517b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 >> 172.21.15.49:0/224170646 conn(0x7f41d0067d90 msgr2=0x7f41d00c78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 shutdown_connections 2024-05-09T08:58:30.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.039+0000 7f41d64a5640 1 -- 172.21.15.49:0/224170646 wait complete. 2024-05-09T08:58:30.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T08:58:30.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T08:58:30.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T08:58:30.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T08:58:30.052 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T08:58:30.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T08:58:30.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T08:58:30.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T08:58:30.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.234+0000 7f1d8521c640 1 Processor -- start 2024-05-09T08:58:30.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.234+0000 7f49dc0c3640 1 Processor -- start 2024-05-09T08:58:30.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.234+0000 7fea4ab1d640 1 Processor -- start 2024-05-09T08:58:30.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.234+0000 7fea4ab1d640 1 -- start start 2024-05-09T08:58:30.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.234+0000 7fe7725d5640 1 Processor -- start 2024-05-09T08:58:30.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f127c838640 1 Processor -- start 2024-05-09T08:58:30.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7fe7725d5640 1 -- start start 2024-05-09T08:58:30.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f127c838640 1 -- start start 2024-05-09T08:58:30.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f1d8521c640 1 -- start start 2024-05-09T08:58:30.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f3e30714640 1 Processor -- start 2024-05-09T08:58:30.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f49dc0c3640 1 -- start start 2024-05-09T08:58:30.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7ffa86b47640 1 Processor -- start 2024-05-09T08:58:30.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7ffa86b47640 1 -- start start 2024-05-09T08:58:30.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.235+0000 7f3e30714640 1 -- start start 2024-05-09T08:58:30.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe7725d5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe76c06bfa0 0x7fe76c0c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe7725d5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe76c06e480 0x7fe76c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.238 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f49dc0c3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 0x7f49d406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f49dc0c3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 0x7f49d4070420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f49dc0c3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49d406b550 con 0x7f49d4159490 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49dc0c3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f49d4159870 con 0x7f49d40c6d90 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49dc0c3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f49d406b830 con 0x7f49d4163040 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f1d8521c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1d80151000 0x7f1d800c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f1d8521c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1d8015a940 0x7f1d8015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f1d8521c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d8015b2f0 con 0x7f1d800c7910 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f1d8521c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1d80157d20 con 0x7f1d80151000 2024-05-09T08:58:30.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f1d8521c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f1d801571a0 con 0x7f1d8015a940 2024-05-09T08:58:30.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f0391bdf640 1 Processor -- start 2024-05-09T08:58:30.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f0391bdf640 1 -- start start 2024-05-09T08:58:30.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa86b47640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49d9e5f640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f49d4159490 0x7f49d406dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49074/0 (socket says 172.21.15.49:49074) 2024-05-09T08:58:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49d965e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 0x7f49d406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49d965e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 0x7f49d406c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45434/0 (socket says 172.21.15.49:45434) 2024-05-09T08:58:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f1d7f59e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1d8015a940 0x7f1d8015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.241 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49d965e640 1 -- 172.21.15.49:0/1791852034 learned_addr learned my addr 172.21.15.49:0/1791852034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f127c838640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f127806bfa0 0x7f12780c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f127c838640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f127806e480 0x7f12780c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f127c838640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1278155440 con 0x7f1278155060 2024-05-09T08:58:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f127c838640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f127815e3f0 con 0x7f127806bfa0 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7f127c838640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f127815e6d0 con 0x7f127806e480 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f1275d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f127806bfa0 0x7f12780c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f1275d9b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f127806bfa0 0x7f12780c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45410/0 (socket says 172.21.15.49:45410) 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f1275d9b640 1 -- 172.21.15.49:0/3920145228 learned_addr learned my addr 172.21.15.49:0/3920145228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f1276d9d640 1 --2- 172.21.15.49:0/3920145228 >> v2:172.21.15.183:3300/0 conn(0x7f127806e480 0x7f12780c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f49da660640 1 --2- 172.21.15.49:0/1791852034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 0x7f49d4070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa85b45640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa85b45640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47468/0 (socket says 172.21.15.49:47468) 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa85b45640 1 -- 172.21.15.49:0/2049375743 learned_addr learned my addr 172.21.15.49:0/2049375743 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e30714640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e2004c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e30714640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3e20046550 0x7f3e200469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e30714640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e20046f20 con 0x7f3e200dd510 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e30714640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3e200e4090 con 0x7f3e20046550 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e30714640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3e200dd8f0 con 0x7f3e2004bee0 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e2e4b0640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f3e200dd510 0x7f3e200455a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47618/0 (socket says 172.21.15.49:47618) 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e2e4b0640 1 -- 172.21.15.49:0/3106837022 learned_addr learned my addr 172.21.15.49:0/3106837022 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8015fe80 con 0x7ffa8006dd30 2024-05-09T08:58:30.245 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7fea488b9640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fea44154b90 0x7fea440cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47588/0 (socket says 172.21.15.49:47588) 2024-05-09T08:58:30.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa800c8780 con 0x7ffa800c8280 2024-05-09T08:58:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7fea4ab1d640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fea44154f70 0x7fea440c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.250 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7f3e2dcaf640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e2004c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7fea488b9640 1 -- 172.21.15.49:0/198881416 learned_addr learned my addr 172.21.15.49:0/198881416 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe7725d5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe76c155440 con 0x7fe76c155060 2024-05-09T08:58:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe7725d5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe76c15e3f0 con 0x7fe76c06bfa0 2024-05-09T08:58:30.251 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe7725d5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe76c15e6d0 con 0x7fe76c06e480 2024-05-09T08:58:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe76bfff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe76c155060 0x7fe76c06b890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49056/0 (socket says 172.21.15.49:49056) 2024-05-09T08:58:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.236+0000 7fe76bfff640 1 -- 172.21.15.49:0/878056233 learned_addr learned my addr 172.21.15.49:0/878056233 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.237+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/878056233 >> v2:172.21.15.49:3300/0 conn(0x7fe76c06bfa0 0x7fe76c0c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea440c6fd0 con 0x7fea44154b90 2024-05-09T08:58:30.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1d77fff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f1d800c7910 0x7f1d8006dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49066/0 (socket says 172.21.15.49:49066) 2024-05-09T08:58:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1d77fff640 1 -- 172.21.15.49:0/3757799531 learned_addr learned my addr 172.21.15.49:0/3757799531 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea44072600 con 0x7fea440c6b80 2024-05-09T08:58:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fea440728e0 con 0x7fea44154f70 2024-05-09T08:58:30.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/3106837022 >> v2:172.21.15.183:3300/0 conn(0x7f3e20046550 0x7f3e200469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f49da660640 1 -- 172.21.15.49:0/1791852034 >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 msgr2=0x7f49d406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1d7ed9d640 1 --2- 172.21.15.49:0/3757799531 >> v2:172.21.15.49:3300/0 conn(0x7f1d80151000 0x7f1d800c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.254 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe770b72640 1 --2- 172.21.15.49:0/878056233 >> v2:172.21.15.183:3300/0 conn(0x7fe76c06e480 0x7fe76c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1275d9b640 1 -- 172.21.15.49:0/3920145228 >> v2:172.21.15.183:3300/0 conn(0x7f127806e480 msgr2=0x7f12780c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1275d9b640 1 --2- 172.21.15.49:0/3920145228 >> v2:172.21.15.183:3300/0 conn(0x7f127806e480 0x7f12780c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1275d9b640 1 -- 172.21.15.49:0/3920145228 >> v1:172.21.15.49:6789/0 conn(0x7f1278155060 legacy=0x7f127806b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1275d9b640 1 -- 172.21.15.49:0/3920145228 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127815e9b0 con 0x7f127806bfa0 2024-05-09T08:58:30.255 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8006ea50 con 0x7ffa80158f10 2024-05-09T08:58:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f49da660640 1 --2- 172.21.15.49:0/1791852034 >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 0x7f49d406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f49da660640 1 -- 172.21.15.49:0/1791852034 >> v1:172.21.15.183:6789/0 conn(0x7f49d4159490 legacy=0x7f49d406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2653492264 0 0) 0x7fe76c155440 con 0x7fe76c155060 2024-05-09T08:58:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe758003520 con 0x7fe76c155060 2024-05-09T08:58:30.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1d7e59c640 1 -- 172.21.15.49:0/3757799531 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906607589 0 0) 0x7f1d68001110 con 0x7f1d800c7910 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f1d7e59c640 1 -- 172.21.15.49:0/3757799531 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d6c005730 con 0x7f1d800c7910 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f0391bdf640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f038c06cfc0 0x7f038c074840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f0391bdf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c074d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f0391bdf640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038c0ca200 con 0x7f038c1592c0 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f0391bdf640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f038c15f0a0 con 0x7f038c06cfc0 2024-05-09T08:58:30.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f0391bdf640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f038c0ca4e0 con 0x7f038c0731b0 2024-05-09T08:58:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f49da660640 1 -- 172.21.15.49:0/1791852034 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d406bb10 con 0x7f49d4163040 2024-05-09T08:58:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fea3bfff640 1 --2- 172.21.15.49:0/198881416 >> v2:172.21.15.183:3300/0 conn(0x7fea44154f70 0x7fea440c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3169197724 0 0) 0x7fe758003520 con 0x7fe76c155060 2024-05-09T08:58:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 >> v2:172.21.15.183:3300/0 conn(0x7fe76c06e480 msgr2=0x7fe76c0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.258 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 --2- 172.21.15.49:0/878056233 >> v2:172.21.15.183:3300/0 conn(0x7fe76c06e480 0x7fe76c0c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f038bfff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f038c06cfc0 0x7f038c074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 >> v2:172.21.15.49:3300/0 conn(0x7fe76c06bfa0 msgr2=0x7fe76c0c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7fe76affd640 1 --2- 172.21.15.49:0/878056233 >> v2:172.21.15.49:3300/0 conn(0x7fe76c06bfa0 0x7fe76c0c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f038bfff640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f038c06cfc0 0x7f038c074840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:48980/0 (socket says 172.21.15.49:48980) 2024-05-09T08:58:30.259 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f038bfff640 1 -- 172.21.15.49:0/4036475917 learned_addr learned my addr 172.21.15.49:0/4036475917 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c15e9b0 con 0x7fe76c155060 2024-05-09T08:58:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fe764001670 con 0x7fe76c155060 2024-05-09T08:58:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f038bfff640 1 -- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 msgr2=0x7f038c074d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f038bfff640 1 --2- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.260 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f038bfff640 1 -- 172.21.15.49:0/4036475917 >> v1:172.21.15.183:6789/0 conn(0x7f038c1592c0 legacy=0x7f038c06b200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T08:58:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 >> v1:172.21.15.183:6789/0 conn(0x7fe76c155060 legacy=0x7fe76c06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7ffa85b45640 1 -- 172.21.15.49:0/2049375743 >> v1:172.21.15.183:6789/0 conn(0x7ffa800c8280 legacy=0x7ffa8006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.261 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f038bfff640 1 -- 172.21.15.49:0/4036475917 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c0ca7c0 con 0x7f038c06cfc0 2024-05-09T08:58:30.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe758003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758004800 0x7fe758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7ffa85b45640 1 -- 172.21.15.49:0/2049375743 >> v1:172.21.15.49:6789/0 conn(0x7ffa8006dd30 legacy=0x7ffa800c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.262 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7ffa85b45640 1 -- 172.21.15.49:0/2049375743 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa800c6c90 con 0x7ffa80158f10 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76bfff640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe758003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7580051c0 0x7fe75800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f1275d9b640 1 --2- 172.21.15.49:0/3920145228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806bfa0 0x7f12780c6c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1278069730 tx=0x7f126c02a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209307b7a773c641 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037101549 0 0) 0x7fea44072600 con 0x7fea440c6b80 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea2c003520 con 0x7fea440c6b80 2024-05-09T08:58:30.263 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065822508 0 0) 0x7fea2c003520 con 0x7fea440c6b80 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 >> v2:172.21.15.183:3300/0 conn(0x7fea44154f70 msgr2=0x7fea440c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 --2- 172.21.15.49:0/198881416 >> v2:172.21.15.183:3300/0 conn(0x7fea44154f70 0x7fea440c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 >> v1:172.21.15.49:6789/0 conn(0x7fea44154b90 legacy=0x7fea440cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea440c5880 con 0x7fea440c6b80 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fea3c0019f0 con 0x7fea440c6b80 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 >> v1:172.21.15.183:6789/0 conn(0x7fea440c6b80 legacy=0x7fea44071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c15e3f0 con 0x7fe758004800 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c15e6d0 con 0x7fe7580051c0 2024-05-09T08:58:30.264 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe75800c010 con 0x7fe7580039b0 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/878056233 >> v2:172.21.15.49:3300/0 conn(0x7fe76c06bfa0 0x7fe76c0c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe770b72640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7580051c0 0x7fe75800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758004800 0x7fe758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f1d7f59e640 1 -- 172.21.15.49:0/3757799531 >> v2:172.21.15.49:3300/0 conn(0x7f1d80151000 msgr2=0x7f1d800c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f1d7f59e640 1 --2- 172.21.15.49:0/3757799531 >> v2:172.21.15.49:3300/0 conn(0x7f1d80151000 0x7f1d800c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f1d7f59e640 1 -- 172.21.15.49:0/3757799531 >> v1:172.21.15.183:6789/0 conn(0x7f1d800c7910 legacy=0x7f1d8006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f1d7f59e640 1 -- 172.21.15.49:0/3757799531 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d80157480 con 0x7f1d8015a940 2024-05-09T08:58:30.265 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.239+0000 7f49da660640 1 --2- 172.21.15.49:0/1791852034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 0x7f49d4070420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009d90 tx=0x7f49cc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1038115fe119780a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f3e2ecb1640 1 -- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 msgr2=0x7f3e2004c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e2004c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f3e2ecb1640 1 -- 172.21.15.49:0/3106837022 >> v1:172.21.15.49:6789/0 conn(0x7f3e200dd510 legacy=0x7f3e200455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.238+0000 7f3e2ecb1640 1 -- 172.21.15.49:0/3106837022 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e200e3510 con 0x7f3e20046550 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea2c004800 0x7fea2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.266 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea2c0051c0 0x7fea2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200469e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e240097a0 tx=0x7f3e2402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe1bace59db7363 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f127559a640 1 -- 172.21.15.49:0/3920145228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c037070 con 0x7f127806bfa0 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76bfff640 1 -- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7580051c0 msgr2=0x7fe75800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76bfff640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7580051c0 0x7fe75800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76bfff640 1 -- 172.21.15.49:0/878056233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758004800 msgr2=0x7fe758004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76bfff640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe758004800 0x7fe758004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7ffa85b45640 1 --2- 172.21.15.49:0/2049375743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa800c6e60 tx=0x7ffa7002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b57d82d7303ee026 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea440728e0 con 0x7fea2c004800 2024-05-09T08:58:30.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c00c070 con 0x7fea2c0039b0 2024-05-09T08:58:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe76bfff640 1 -- 172.21.15.49:0/878056233 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe758003520 con 0x7fe7580039b0 2024-05-09T08:58:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fe770b72640 1 -- 172.21.15.49:0/878056233 reap_dead start 2024-05-09T08:58:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f127559a640 1 -- 172.21.15.49:0/3920145228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126c032640 con 0x7f127806bfa0 2024-05-09T08:58:30.269 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fea2c00c270 con 0x7fea2c0051c0 2024-05-09T08:58:30.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f49d8e5d640 1 -- 172.21.15.49:0/1791852034 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc031600 con 0x7f49d4163040 2024-05-09T08:58:30.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f49d8e5d640 1 -- 172.21.15.49:0/1791852034 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc0317a0 con 0x7f49d4163040 2024-05-09T08:58:30.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f49d8e5d640 1 -- 172.21.15.49:0/1791852034 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc031b70 con 0x7f49d4163040 2024-05-09T08:58:30.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea488b9640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.270 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f3e2d4ae640 1 -- 172.21.15.49:0/3106837022 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24037070 con 0x7f3e20046550 2024-05-09T08:58:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7ffa84b43640 1 -- 172.21.15.49:0/2049375743 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70037070 con 0x7ffa80158f10 2024-05-09T08:58:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7f3e2d4ae640 1 -- 172.21.15.49:0/3106837022 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e24032440 con 0x7f3e20046550 2024-05-09T08:58:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea490ba640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea2c0051c0 0x7fea2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7fea3bfff640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea2c004800 0x7fea2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.240+0000 7ffa84b43640 1 -- 172.21.15.49:0/2049375743 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa70032440 con 0x7ffa80158f10 2024-05-09T08:58:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea488b9640 1 -- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea2c0051c0 msgr2=0x7fea2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea488b9640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea2c0051c0 0x7fea2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea488b9640 1 -- 172.21.15.49:0/198881416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea2c004800 msgr2=0x7fea2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea488b9640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea2c004800 0x7fea2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea488b9640 1 -- 172.21.15.49:0/198881416 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea440c5880 con 0x7fea2c0039b0 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea490ba640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea2c0051c0 0x7fea2c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fea490ba640 1 -- 172.21.15.49:0/198881416 reap_dead start 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f1d7f59e640 1 --2- 172.21.15.49:0/3757799531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d8015adb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d80071f40 tx=0x7f1d78008740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52adf5cfeb13d988 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa84b43640 1 -- 172.21.15.49:0/2049375743 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70032810 con 0x7ffa80158f10 2024-05-09T08:58:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7fe76bfff640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe758003dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c240 tx=0x7fe758005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d4d2224d0f7e37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f3e2d4ae640 1 -- 172.21.15.49:0/3106837022 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24032810 con 0x7f3e20046550 2024-05-09T08:58:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f1d7e59c640 1 -- 172.21.15.49:0/3757799531 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68001840 con 0x7f1d8015a940 2024-05-09T08:58:30.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f1d7e59c640 1 -- 172.21.15.49:0/3757799531 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d78035070 con 0x7f1d8015a940 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f3e30714640 1 -- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 msgr2=0x7f3e200469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f3e30714640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200469e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e240097a0 tx=0x7f3e2402a230 comp rx=0 tx=0).stop 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 -- 172.21.15.49:0/3920145228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806bfa0 msgr2=0x7f12780c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 --2- 172.21.15.49:0/3920145228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806bfa0 0x7f12780c6c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1278069730 tx=0x7f126c02a320 comp rx=0 tx=0).stop 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 -- 172.21.15.49:0/3920145228 shutdown_connections 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 --2- 172.21.15.49:0/3920145228 >> v2:172.21.15.183:3300/0 conn(0x7f127806e480 0x7f12780c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 --2- 172.21.15.49:0/3920145228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806bfa0 0x7f12780c6c90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f127c838640 1 -- 172.21.15.49:0/3920145228 >> 172.21.15.49:0/3920145228 conn(0x7f12780cc290 msgr2=0x7f12780cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 msgr2=0x7ffa8015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa86b47640 1 --2- 172.21.15.49:0/2049375743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa800c6e60 tx=0x7ffa7002a180 comp rx=0 tx=0).stop 2024-05-09T08:58:30.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7f038bfff640 1 --2- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c074840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f037c009870 tx=0x7f037c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7946e92b57d26db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 shutdown_connections 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa86b47640 1 --2- 172.21.15.49:0/2049375743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa80158f10 0x7ffa8015f940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.241+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 >> 172.21.15.49:0/2049375743 conn(0x7ffa800cc0f0 msgr2=0x7ffa800cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 shutdown_connections 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7ffa86b47640 1 -- 172.21.15.49:0/2049375743 wait complete. 2024-05-09T08:58:30.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 -- 172.21.15.49:0/3106837022 shutdown_connections 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200469e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 --2- 172.21.15.49:0/3106837022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e2004c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 -- 172.21.15.49:0/3106837022 >> 172.21.15.49:0/3106837022 conn(0x7f3e20050600 msgr2=0x7f3e20050a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fea488b9640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea2c003dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea34001810 tx=0x7fea2c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59cbfab2564131ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34001460 con 0x7fea2c0039b0 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3402e070 con 0x7fea2c0039b0 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fea3b7fe640 1 -- 172.21.15.49:0/198881416 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3402add0 con 0x7fea2c0039b0 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 msgr2=0x7fea2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fea4ab1d640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea2c003dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea34001810 tx=0x7fea2c005830 comp rx=0 tx=0).stop 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764007d30 con 0x7fe7580039b0 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe764005ad0 con 0x7fe7580039b0 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7fe76affd640 1 -- 172.21.15.49:0/878056233 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764007070 con 0x7fe7580039b0 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fe7725d5640 1 -- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 msgr2=0x7fe758003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fe7725d5640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe758003dd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c240 tx=0x7fe758005830 comp rx=0 tx=0).stop 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d7e59c640 1 -- 172.21.15.49:0/3757799531 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800a500 con 0x7f1d8015a940 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 -- 172.21.15.49:0/3757799531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 msgr2=0x7f1d8015adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 --2- 172.21.15.49:0/3757799531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d8015adb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1d80071f40 tx=0x7f1d78008740 comp rx=0 tx=0).stop 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 -- 172.21.15.49:0/3757799531 shutdown_connections 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 --2- 172.21.15.49:0/3757799531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d8015adb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 --2- 172.21.15.49:0/3757799531 >> v2:172.21.15.49:3300/0 conn(0x7f1d80151000 0x7f1d800c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 -- 172.21.15.49:0/3757799531 >> 172.21.15.49:0/3757799531 conn(0x7f1d800cc290 msgr2=0x7f1d800cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f1d8521c640 1 -- 172.21.15.49:0/3757799531 shutdown_connections 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7f1d8521c640 1 -- 172.21.15.49:0/3757799531 wait complete. 2024-05-09T08:58:30.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 -- 172.21.15.49:0/1791852034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 msgr2=0x7f49d4070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 --2- 172.21.15.49:0/1791852034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 0x7f49d4070420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009d90 tx=0x7f49cc02a5d0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 -- 172.21.15.49:0/1791852034 shutdown_connections 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 --2- 172.21.15.49:0/1791852034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4163040 0x7f49d4070420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 --2- 172.21.15.49:0/1791852034 >> v2:172.21.15.49:3300/0 conn(0x7f49d40c6d90 0x7f49d406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 -- 172.21.15.49:0/1791852034 >> 172.21.15.49:0/1791852034 conn(0x7f49d40cc290 msgr2=0x7f49d40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 -- 172.21.15.49:0/1791852034 shutdown_connections 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f49dc0c3640 1 -- 172.21.15.49:0/1791852034 wait complete. 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 -- 172.21.15.49:0/3106837022 shutdown_connections 2024-05-09T08:58:30.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f3e30714640 1 -- 172.21.15.49:0/3106837022 wait complete. 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f038b7fe640 1 -- 172.21.15.49:0/4036475917 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c037070 con 0x7f038c06cfc0 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f038b7fe640 1 -- 172.21.15.49:0/4036475917 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c032440 con 0x7f038c06cfc0 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f038b7fe640 1 -- 172.21.15.49:0/4036475917 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c032810 con 0x7f038c06cfc0 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7f0391bdf640 1 -- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 msgr2=0x7f038c074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7f0391bdf640 1 --2- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c074840 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f037c009870 tx=0x7f037c02a210 comp rx=0 tx=0).stop 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f127c838640 1 -- 172.21.15.49:0/3920145228 shutdown_connections 2024-05-09T08:58:30.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.242+0000 7f127c838640 1 -- 172.21.15.49:0/3920145228 wait complete. 2024-05-09T08:58:30.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7ffa86b47640 1 Processor -- start 2024-05-09T08:58:30.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 shutdown_connections 2024-05-09T08:58:30.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fea4ab1d640 1 --2- 172.21.15.49:0/198881416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea2c003dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7f127c838640 1 Processor -- start 2024-05-09T08:58:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.243+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 >> 172.21.15.49:0/198881416 conn(0x7fea44074840 msgr2=0x7fea44074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7ffa86b47640 1 -- start start 2024-05-09T08:58:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f3e30714640 1 Processor -- start 2024-05-09T08:58:30.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fe7725d5640 1 -- 172.21.15.49:0/878056233 shutdown_connections 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 shutdown_connections 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fea4ab1d640 1 -- 172.21.15.49:0/198881416 wait complete. 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 -- 172.21.15.49:0/4036475917 shutdown_connections 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 --2- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c074d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 --2- 172.21.15.49:0/4036475917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c074840 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 -- 172.21.15.49:0/4036475917 >> 172.21.15.49:0/4036475917 conn(0x7f038c06e810 msgr2=0x7f038c0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f127c838640 1 -- start start 2024-05-09T08:58:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f49dc0c3640 1 Processor -- start 2024-05-09T08:58:30.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f49dc0c3640 1 -- start start 2024-05-09T08:58:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fe7725d5640 1 --2- 172.21.15.49:0/878056233 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe758003dd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fe7725d5640 1 -- 172.21.15.49:0/878056233 >> 172.21.15.49:0/878056233 conn(0x7fe76c0cc290 msgr2=0x7fe76c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f1d8521c640 1 Processor -- start 2024-05-09T08:58:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fe7725d5640 1 -- 172.21.15.49:0/878056233 shutdown_connections 2024-05-09T08:58:30.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7fe7725d5640 1 -- 172.21.15.49:0/878056233 wait complete. 2024-05-09T08:58:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 -- 172.21.15.49:0/4036475917 shutdown_connections 2024-05-09T08:58:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa86b47640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 0x7ffa801ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f1d8521c640 1 -- start start 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f3e30714640 1 -- start start 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e200e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e200dd510 0x7f3e200e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3e200e3db0 con 0x7f3e200dd510 2024-05-09T08:58:30.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa86b47640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa800c8280 0x7ffa800c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e200e3ad0 con 0x7f3e20046550 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f3e30714640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3e20046030 con 0x7f3e2004bee0 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa86b47640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa80158f10 0x7ffa8015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa85b45640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 0x7ffa801ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa85b45640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 0x7ffa801ea5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47512/0 (socket says 172.21.15.49:47512) 2024-05-09T08:58:30.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa85b45640 1 -- 172.21.15.49:0/2216519641 learned_addr learned my addr 172.21.15.49:0/2216519641 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f49dc0c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 0x7f49d4156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8006e770 con 0x7ffa800c8280 2024-05-09T08:58:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801592f0 con 0x7ffa80158f10 2024-05-09T08:58:30.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49dc0c3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 0x7f49d415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa86346640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa80158f10 0x7ffa8015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f3e2e4b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f3e2e4b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200de4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49000/0 (socket says 172.21.15.49:49000) 2024-05-09T08:58:30.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f3e2e4b0640 1 -- 172.21.15.49:0/1512062553 learned_addr learned my addr 172.21.15.49:0/1512062553 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8006d870 con 0x7ffa8006dd30 2024-05-09T08:58:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49dc0c3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49d4163040 0x7f49d40ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49d9e5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 0x7f49d4156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49d965e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 0x7f49d415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7fea4ab1d640 1 Processor -- start 2024-05-09T08:58:30.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49d965e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 0x7f49d415f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47520/0 (socket says 172.21.15.49:47520) 2024-05-09T08:58:30.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49d9e5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 0x7f49d4156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49030/0 (socket says 172.21.15.49:49030) 2024-05-09T08:58:30.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e200dd510 0x7f3e200e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49d965e640 1 -- 172.21.15.49:0/4075754576 learned_addr learned my addr 172.21.15.49:0/4075754576 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49dc0c3640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4163480 con 0x7f49d4163040 2024-05-09T08:58:30.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85344640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa800c8280 0x7ffa800c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7fea4ab1d640 1 -- start start 2024-05-09T08:58:30.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49da660640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49d4163040 0x7f49d40ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49dc0c3640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49d406c0d0 con 0x7f49d40c6d90 2024-05-09T08:58:30.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f49dc0c3640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f49d406e1d0 con 0x7f49d4159490 2024-05-09T08:58:30.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.244+0000 7f0391bdf640 1 -- 172.21.15.49:0/4036475917 wait complete. 2024-05-09T08:58:30.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127c838640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 0x7f12781f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 0x7f12781f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 0x7f12781f02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45448/0 (socket says 172.21.15.49:45448) 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127659c640 1 -- 172.21.15.49:0/3838511117 learned_addr learned my addr 172.21.15.49:0/3838511117 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127c838640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278155060 0x7f12780c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127c838640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12781f45e0 0x7f12780c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.245+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f12780c83a0 con 0x7f127806e480 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f127815ef70 con 0x7f12781f45e0 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f12780c8520 con 0x7f1278155060 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f1276d9d640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12781f45e0 0x7f12780c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127659c640 1 -- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278155060 msgr2=0x7f12780c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127659c640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1278155060 0x7f12780c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127659c640 1 -- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12781f45e0 msgr2=0x7f12780c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127659c640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12781f45e0 0x7f12780c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f127659c640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127815d370 con 0x7f127806e480 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f0391bdf640 1 Processor -- start 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f0391bdf640 1 -- start start 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7f3e2e4b0640 1 -- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 msgr2=0x7f3e200e5be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2e4b0640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3e2004bee0 0x7f3e200e5be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2e4b0640 1 -- 172.21.15.49:0/1512062553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e200dd510 msgr2=0x7f3e200e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85b45640 1 -- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa80158f10 msgr2=0x7ffa8015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2e4b0640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e200dd510 0x7f3e200e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2e4b0640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2017a5e0 con 0x7f3e20046550 2024-05-09T08:58:30.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85b45640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa80158f10 0x7ffa8015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85b45640 1 -- 172.21.15.49:0/2216519641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa800c8280 msgr2=0x7ffa800c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 0x7f1d801f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85b45640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa800c8280 0x7ffa800c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.246+0000 7ffa85b45640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801eaae0 con 0x7ffa8006dd30 2024-05-09T08:58:30.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7fe7725d5640 1 Processor -- start 2024-05-09T08:58:30.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3e200dd510 0x7f3e200e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:30.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7ffa85344640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ffa800c8280 0x7ffa800c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f49da660640 1 -- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 msgr2=0x7f49d415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d80151000 0x7f1d8014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7fe7725d5640 1 -- start start 2024-05-09T08:58:30.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7ffa85b45640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 0x7ffa801ea5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009910 tx=0x7ffa700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d7ed9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 0x7f1d801f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d7ed9d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 0x7f1d801f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47530/0 (socket says 172.21.15.49:47530) 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d7ed9d640 1 -- 172.21.15.49:0/2263321515 learned_addr learned my addr 172.21.15.49:0/2263321515 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f3e2e4b0640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200de4d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c3c0 tx=0x7f3e1800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d80146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8015a420 con 0x7f1d80151000 2024-05-09T08:58:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d80157a40 con 0x7f1d8015a940 2024-05-09T08:58:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1d800c71d0 con 0x7f1d800c7910 2024-05-09T08:58:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70037040 con 0x7ffa8006dd30 2024-05-09T08:58:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d77fff640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d80151000 0x7f1d8014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18017070 con 0x7f3e20046550 2024-05-09T08:58:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.247+0000 7f49da660640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 0x7f49d415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa70004230 con 0x7ffa8006dd30 2024-05-09T08:58:30.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea441f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7f59e640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d80146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801ead70 con 0x7ffa8006dd30 2024-05-09T08:58:30.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f038c1592c0 0x7f038c1f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f038c15f530 con 0x7f038c1592c0 2024-05-09T08:58:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f038c1596a0 con 0x7f038c06cfc0 2024-05-09T08:58:30.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 0x7fea440c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0391bdf640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f038c15f6b0 con 0x7f038c0731b0 2024-05-09T08:58:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801f8740 con 0x7ffa8006dd30 2024-05-09T08:58:30.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 -- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 msgr2=0x7f1d80146a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44155f00 0x7fea44156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c1504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49044/0 (socket says 172.21.15.49:49044) 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 -- 172.21.15.49:0/1567691706 learned_addr learned my addr 172.21.15.49:0/1567691706 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea3bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 0x7fea440c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea3bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 0x7fea440c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47554/0 (socket says 172.21.15.49:47554) 2024-05-09T08:58:30.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea3bfff640 1 -- 172.21.15.49:0/2321869220 learned_addr learned my addr 172.21.15.49:0/2321869220 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d8015a940 0x7f1d80146a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 -- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d80151000 msgr2=0x7f1d8014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1d80151000 0x7f1d8014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8014dbe0 con 0x7f1d800c7910 2024-05-09T08:58:30.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7003c730 con 0x7ffa8006dd30 2024-05-09T08:58:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f038bfff640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fea440c6120 con 0x7fea44155f00 2024-05-09T08:58:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea440c5e40 con 0x7fea2c0039b0 2024-05-09T08:58:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fea4415af40 con 0x7fea44157530 2024-05-09T08:58:30.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f1d7ed9d640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 0x7f1d801f1600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000d620 tx=0x7f1d7000dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f49da660640 1 -- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 msgr2=0x7f49d4156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 -- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 msgr2=0x7f038c150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f038c0731b0 0x7f038c150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 -- 172.21.15.49:0/1567691706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f038c1592c0 msgr2=0x7f038c1f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f038c1592c0 0x7f038c1f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f0390bdd640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c1fe510 con 0x7f038c06cfc0 2024-05-09T08:58:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f127659c640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 0x7f12781f02c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f126800cb20 tx=0x7f1268002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f49da660640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 0x7f49d4156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2017a880 con 0x7f3e20046550 2024-05-09T08:58:30.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e20047eb0 con 0x7f3e20046550 2024-05-09T08:58:30.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e18005da0 con 0x7f3e20046550 2024-05-09T08:58:30.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe76c1f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 0x7fe76c1ee410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f49da660640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d4155560 con 0x7f49d4163040 2024-05-09T08:58:30.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f0390bdd640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c1504c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f038400bab0 tx=0x7f038400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e180072d0 con 0x7f3e20046550 2024-05-09T08:58:30.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea490ba640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44155f00 0x7fea44156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe76c1539f0 0x7fe76c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268003b70 con 0x7f127806e480 2024-05-09T08:58:30.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268003d10 con 0x7f127806e480 2024-05-09T08:58:30.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126800a930 con 0x7f127806e480 2024-05-09T08:58:30.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe76b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 0x7fe76c1ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe76b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 0x7fe76c1ee410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45512/0 (socket says 172.21.15.49:45512) 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe76c1f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe76bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe76c1f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47566/0 (socket says 172.21.15.49:47566) 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 -- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 msgr2=0x7fea441f0c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea2c0039b0 0x7fea441f0c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 -- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44155f00 msgr2=0x7fea44156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fea44155f00 0x7fea44156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea44157ea0 con 0x7fea44157530 2024-05-09T08:58:30.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe76b7fe640 1 -- 172.21.15.49:0/184075524 learned_addr learned my addr 172.21.15.49:0/184075524 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c0c6c90 con 0x7fe76c0c8e60 2024-05-09T08:58:30.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c15ef70 con 0x7fe76c1539f0 2024-05-09T08:58:30.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe76c1539f0 0x7fe76c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe76c0c6e10 con 0x7fe7580039b0 2024-05-09T08:58:30.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e10001030 con 0x7f3e20046550 2024-05-09T08:58:30.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7fea3bfff640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 0x7fea440c9cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea40011ad0 tx=0x7fea40011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000b620 con 0x7f1d800c7910 2024-05-09T08:58:30.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d70019070 con 0x7f1d800c7910 2024-05-09T08:58:30.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384019070 con 0x7f038c06cfc0 2024-05-09T08:58:30.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c0c9010 con 0x7f038c06cfc0 2024-05-09T08:58:30.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000b9e0 con 0x7f1d800c7910 2024-05-09T08:58:30.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7fe76b7fe640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 msgr2=0x7fe76c1f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7580039b0 0x7fe76c1f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7fe76b7fe640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe76c1539f0 msgr2=0x7fe76c1f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe76c1539f0 0x7fe76c1f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7fe76b7fe640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c1ee950 con 0x7fe76c0c8e60 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d801f1b40 con 0x7f1d800c7910 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d800c5650 con 0x7f1d800c7910 2024-05-09T08:58:30.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.248+0000 7f49d965e640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49d4159490 0x7f49d415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f49d9e5f640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49d40c6d90 0x7f49d4156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7f49da660640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49d4163040 0x7f49d40ca8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f49cc031000 tx=0x7f49cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa8015e990 con 0x7ffa8006dd30 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c1fe7b0 con 0x7f038c06cfc0 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038400cc90 con 0x7f038c06cfc0 2024-05-09T08:58:30.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384014680 con 0x7f038c06cfc0 2024-05-09T08:58:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12780c4cb0 con 0x7f127806e480 2024-05-09T08:58:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.249+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12781f3b00 con 0x7f127806e480 2024-05-09T08:58:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.250+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1278072140 con 0x7f127806e480 2024-05-09T08:58:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4000fbb0 con 0x7fea44157530 2024-05-09T08:58:30.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f036c001030 con 0x7f038c06cfc0 2024-05-09T08:58:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea4000fd50 con 0x7fea44157530 2024-05-09T08:58:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea44158160 con 0x7fea44157530 2024-05-09T08:58:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea44155700 con 0x7fea44157530 2024-05-09T08:58:30.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc040070 con 0x7f49d4163040 2024-05-09T08:58:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa7003a070 con 0x7ffa8006dd30 2024-05-09T08:58:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc030b20 con 0x7f49d4163040 2024-05-09T08:58:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40014660 con 0x7fea44157530 2024-05-09T08:58:30.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.251+0000 7fe76b7fe640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 0x7fe76c1ee410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe754011830 tx=0x7fe754011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d8004f660 con 0x7f1d800c7910 2024-05-09T08:58:30.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754019070 con 0x7fe76c0c8e60 2024-05-09T08:58:30.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7ffa767fc640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa6406bf00 0x7ffa6406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e1801a070 con 0x7f3e20046550 2024-05-09T08:58:30.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f126800aad0 con 0x7f127806e480 2024-05-09T08:58:30.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc045460 con 0x7f49d4163040 2024-05-09T08:58:30.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f49dc0c3640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d4155840 con 0x7f49d4163040 2024-05-09T08:58:30.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.252+0000 7f49dc0c3640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d4155e00 con 0x7f49d4163040 2024-05-09T08:58:30.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49d4160480 con 0x7f49d4163040 2024-05-09T08:58:30.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f038401c070 con 0x7f038c06cfc0 2024-05-09T08:58:30.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f3e1effd640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e0c06bf00 0x7f3e0c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d7000bb80 con 0x7f1d800c7910 2024-05-09T08:58:30.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1d777fe640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1d6006bf30 0x7f1d6006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe754004440 con 0x7fe76c0c8e60 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1266ffd640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f125806bf00 0x7f125806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f126808b790 con 0x7f127806e480 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1266ffd640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1258074380 0x7f1258076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1258076ef0 con 0x7f1258074380 2024-05-09T08:58:30.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.253+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f126808bb80 con 0x7f127806e480 2024-05-09T08:58:30.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea40012040 con 0x7fea44157530 2024-05-09T08:58:30.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c0c93c0 con 0x7fe76c0c8e60 2024-05-09T08:58:30.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea24001030 con 0x7fea44157530 2024-05-09T08:58:30.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c0c7810 con 0x7fe76c0c8e60 2024-05-09T08:58:30.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f1d77fff640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1d6006bf30 0x7f1d6006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f1275d9b640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f125806bf00 0x7f125806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7ffa700ab730 con 0x7ffa8006dd30 2024-05-09T08:58:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75400f9c0 con 0x7fe76c0c8e60 2024-05-09T08:58:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f03897fa640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f037806be30 0x7f037806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea397fa640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fea3006bf00 0x7fea3006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea488b9640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fea3006bf00 0x7fea3006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7fea4008b630 con 0x7fea44157530 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa767fc640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7ffa640743e0 0x7ffa64076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa70032440 con 0x7ffa640743e0 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ffa700b2050 con 0x7ffa8006dd30 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa85344640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa6406bf00 0x7ffa6406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7ffa86346640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7ffa640743e0 0x7ffa64076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7ffa85344640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa6406bf00 0x7ffa6406e3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa800cbbb0 tx=0x7ffa7c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f3e2dcaf640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e0c06bf00 0x7f3e0c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f3e1808ad00 con 0x7f3e20046550 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e1effd640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f3e0c0743e0 0x7f3e0c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e18005da0 con 0x7f3e0c0743e0 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3e1808b0f0 con 0x7f3e20046550 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f038408ad10 con 0x7f038c06cfc0 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f03897fa640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f0378074310 0x7f03780767b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f038bfff640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f037806be30 0x7f037806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0378076e80 con 0x7f0378074310 2024-05-09T08:58:30.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f038408d370 con 0x7f038c06cfc0 2024-05-09T08:58:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f03913de640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f0378074310 0x7f03780767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.254+0000 7f1276d9d640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1258074380 0x7f1258076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f1275d9b640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f125806bf00 0x7f125806e3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f126c034000 tx=0x7f126c02c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f1276d9d640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1258074380 0x7f1258076820 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49cc039040 con 0x7f49d4163040 2024-05-09T08:58:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e2dcaf640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e0c06bf00 0x7f3e0c06e3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e28064d50 tx=0x7f3e2806ae90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f3e0c0743e0 0x7f3e0c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea397fa640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fea30074380 0x7fea30076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea40024e80 con 0x7fea30074380 2024-05-09T08:58:30.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fea40092050 con 0x7fea44157530 2024-05-09T08:58:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7f03913de640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f0378074310 0x7f03780767b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.255+0000 7fea490ba640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fea30074380 0x7fea30076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49ca7fc640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49b806bf00 0x7f49b806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7ffa86346640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7ffa640743e0 0x7ffa64076880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d77fff640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1d6006bf30 0x7f1d6006e3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d800c4dc0 tx=0x7f1d680047e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f1d7008b240 con 0x7f1d800c7910 2024-05-09T08:58:30.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f038bfff640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f037806be30 0x7f037806e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f037c034000 tx=0x7f037c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7fea488b9640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fea3006bf00 0x7fea3006e3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea34032000 tx=0x7fea34007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7f49cc0acbd0 con 0x7f49d4163040 2024-05-09T08:58:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49d9e5f640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49b806bf00 0x7f49b806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d777fe640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7f1d60074410 0x7f1d600768b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f3e2ecb1640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f3e0c0743e0 0x7f3e0c076880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49ca7fc640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f49b80743e0 0x7f49b8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d70019070 con 0x7f1d60074410 2024-05-09T08:58:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7fea490ba640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fea30074380 0x7fea30076820 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f1d70092050 con 0x7f1d800c7910 2024-05-09T08:58:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49cc043e80 con 0x7f49b80743e0 2024-05-09T08:58:30.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f49cc0acfc0 con 0x7f49d4163040 2024-05-09T08:58:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1258076ef0 con 0x7f1258074380 2024-05-09T08:58:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f49d965e640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f49b80743e0 0x7f49b8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7f1d7f59e640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7f1d60074410 0x7f1d600768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.256+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa70032440 con 0x7ffa640743e0 2024-05-09T08:58:30.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7f49d9e5f640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49b806bf00 0x7f49b806e3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000f60 tx=0x7f49d0000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe750001030 con 0x7fe76c0c8e60 2024-05-09T08:58:30.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea40024e80 con 0x7fea30074380 2024-05-09T08:58:30.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0378076e80 con 0x7f0378074310 2024-05-09T08:58:30.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7f49d965e640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f49b80743e0 0x7f49b8076880 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e18005da0 con 0x7f3e0c0743e0 2024-05-09T08:58:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.257+0000 7f1d7f59e640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7f1d60074410 0x7f1d600768b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.258+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49cc043e80 con 0x7f49b80743e0 2024-05-09T08:58:30.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.258+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== osd.2 v2:172.21.15.49:6808/282207482 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d70019070 con 0x7f1d60074410 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.258+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe75401c070 con 0x7fe76c0c8e60 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.259+0000 7fe768ff9640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe74c06bf00 0x7fe74c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.259+0000 7fe76bfff640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe74c06bf00 0x7fe74c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.259+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5886+0+0 (secure 0 0 0) 0x7fe75408b470 con 0x7fe76c0c8e60 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.261+0000 7fe76bfff640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe74c06bf00 0x7fe74c06e3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe764032000 tx=0x7fe764042040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.261+0000 7fe768ff9640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.261+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe74c076f50 con 0x7fe74c0743e0 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.261+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe75408b860 con 0x7fe76c0c8e60 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.262+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.262+0000 7fe770b72640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 msgr2=0x7fe74c076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-09T08:58:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.262+0000 7fe770b72640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 msgr2=0x7fe74c076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T08:58:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.262+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 crc :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T08:58:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.262+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T08:58:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.278+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa8004f5f0 con 0x7ffa640743e0 2024-05-09T08:58:30.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa767fc640 1 -- 172.21.15.49:0/2216519641 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa8004f5f0 con 0x7ffa640743e0 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7ffa640743e0 msgr2=0x7ffa64076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7ffa640743e0 0x7ffa64076880 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa6406bf00 msgr2=0x7ffa6406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ffa6406bf00 0x7ffa6406e3c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa800cbbb0 tx=0x7ffa7c009040 comp rx=0 tx=0).stop 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 msgr2=0x7ffa801ea5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 --2- 172.21.15.49:0/2216519641 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8006dd30 0x7ffa801ea5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009910 tx=0x7ffa700046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86346640 1 -- 172.21.15.49:0/2216519641 reap_dead start 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 shutdown_connections 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.279+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 >> 172.21.15.49:0/2216519641 conn(0x7ffa800cc0f0 msgr2=0x7ffa8015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.280+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 shutdown_connections 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.280+0000 7ffa86b47640 1 -- 172.21.15.49:0/2216519641 wait complete. 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.290+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f127804f660 con 0x7f1258074380 2024-05-09T08:58:30.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.290+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 --> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d80070570 con 0x7f1d60074410 2024-05-09T08:58:30.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.290+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f036c002dd0 con 0x7f0378074310 2024-05-09T08:58:30.351 INFO:teuthology.orchestra.run.smithi049.stdout:55834574881 2024-05-09T08:58:30.352 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:58:30.352 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.291+0000 7f1d777fe640 1 -- 172.21.15.49:0/2263321515 <== osd.2 v2:172.21.15.49:6808/282207482 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d80070570 con 0x7f1d60074410 2024-05-09T08:58:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.291+0000 7f1266ffd640 1 -- 172.21.15.49:0/3838511117 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f127804f660 con 0x7f1258074380 2024-05-09T08:58:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.291+0000 7f03897fa640 1 -- 172.21.15.49:0/1567691706 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f036c002dd0 con 0x7f0378074310 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.291+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7f1d60074410 msgr2=0x7f1d600768b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.291+0000 7f1d8521c640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7f1d60074410 0x7f1d600768b0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1d6006bf30 msgr2=0x7f1d6006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1d6006bf30 0x7f1d6006e3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1d800c4dc0 tx=0x7f1d680047e0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 msgr2=0x7f1d801f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 --2- 172.21.15.49:0/2263321515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d800c7910 0x7f1d801f1600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000d620 tx=0x7f1d7000dae0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d7f59e640 1 -- 172.21.15.49:0/2263321515 reap_dead start 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 shutdown_connections 2024-05-09T08:58:30.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 >> 172.21.15.49:0/2263321515 conn(0x7f1d800cc290 msgr2=0x7f1d801555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 shutdown_connections 2024-05-09T08:58:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1d8521c640 1 -- 172.21.15.49:0/2263321515 wait complete. 2024-05-09T08:58:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e10002dd0 con 0x7f3e0c0743e0 2024-05-09T08:58:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f0378074310 msgr2=0x7f03780767b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.355 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f0378074310 0x7f03780767b0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1258074380 msgr2=0x7f1258076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7f1258074380 0x7f1258076820 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f125806bf00 msgr2=0x7f125806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f125806bf00 0x7f125806e3c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f126c034000 tx=0x7f126c02c000 comp rx=0 tx=0).stop 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 msgr2=0x7f12781f02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 --2- 172.21.15.49:0/3838511117 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f127806e480 0x7f12781f02c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f126800cb20 tx=0x7f1268002100 comp rx=0 tx=0).stop 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f1276d9d640 1 -- 172.21.15.49:0/3838511117 reap_dead start 2024-05-09T08:58:30.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49d404f660 con 0x7f49b80743e0 2024-05-09T08:58:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f3e1effd640 1 -- 172.21.15.49:0/1512062553 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3e10002dd0 con 0x7f3e0c0743e0 2024-05-09T08:58:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 shutdown_connections 2024-05-09T08:58:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 >> 172.21.15.49:0/3838511117 conn(0x7f12780cc290 msgr2=0x7f12781521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 shutdown_connections 2024-05-09T08:58:30.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f127c838640 1 -- 172.21.15.49:0/3838511117 wait complete. 2024-05-09T08:58:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f037806be30 msgr2=0x7f037806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f037806be30 0x7f037806e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f037c034000 tx=0x7f037c02e000 comp rx=0 tx=0).stop 2024-05-09T08:58:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 msgr2=0x7f038c1504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f0391bdf640 1 --2- 172.21.15.49:0/1567691706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f038c06cfc0 0x7f038c1504c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f038400bab0 tx=0x7f038400bf70 comp rx=0 tx=0).stop 2024-05-09T08:58:30.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.292+0000 7f49ca7fc640 1 -- 172.21.15.49:0/4075754576 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f49d404f660 con 0x7f49b80743e0 2024-05-09T08:58:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f03913de640 1 -- 172.21.15.49:0/1567691706 reap_dead start 2024-05-09T08:58:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 shutdown_connections 2024-05-09T08:58:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 >> 172.21.15.49:0/1567691706 conn(0x7f038c06e810 msgr2=0x7f038c0cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.359 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f3e0c0743e0 msgr2=0x7f3e0c076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f3e0c0743e0 0x7f3e0c076880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 shutdown_connections 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e0c06bf00 msgr2=0x7f3e0c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3e0c06bf00 0x7f3e0c06e3c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e28064d50 tx=0x7f3e2806ae90 comp rx=0 tx=0).stop 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 msgr2=0x7f3e200de4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 --2- 172.21.15.49:0/1512062553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e20046550 0x7f3e200de4d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c3c0 tx=0x7f3e1800c880 comp rx=0 tx=0).stop 2024-05-09T08:58:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f0391bdf640 1 -- 172.21.15.49:0/1567691706 wait complete. 2024-05-09T08:58:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e2ecb1640 1 -- 172.21.15.49:0/1512062553 reap_dead start 2024-05-09T08:58:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 shutdown_connections 2024-05-09T08:58:30.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 >> 172.21.15.49:0/1512062553 conn(0x7f3e20050600 msgr2=0x7f3e200dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f49b80743e0 msgr2=0x7f49b8076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f49b7fff640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f49b80743e0 0x7f49b8076880 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 shutdown_connections 2024-05-09T08:58:30.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f3e30714640 1 -- 172.21.15.49:0/1512062553 wait complete. 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49b806bf00 msgr2=0x7f49b806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.293+0000 7f49b7fff640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f49b806bf00 0x7f49b806e3c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000f60 tx=0x7f49d0000ed0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49d4163040 msgr2=0x7f49d40ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 --2- 172.21.15.49:0/4075754576 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f49d4163040 0x7f49d40ca8e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f49cc031000 tx=0x7f49cc004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49da660640 1 -- 172.21.15.49:0/4075754576 reap_dead start 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 shutdown_connections 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 >> 172.21.15.49:0/4075754576 conn(0x7f49d40cc290 msgr2=0x7f49d4157600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea24002dd0 con 0x7fea30074380 2024-05-09T08:58:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 shutdown_connections 2024-05-09T08:58:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7f49b7fff640 1 -- 172.21.15.49:0/4075754576 wait complete. 2024-05-09T08:58:30.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.294+0000 7fea397fa640 1 -- 172.21.15.49:0/2321869220 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea24002dd0 con 0x7fea30074380 2024-05-09T08:58:30.365 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fea30074380 msgr2=0x7fea30076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7fea30074380 0x7fea30076820 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fea3006bf00 msgr2=0x7fea3006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fea3006bf00 0x7fea3006e3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fea34032000 tx=0x7fea34007000 comp rx=0 tx=0).stop 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 msgr2=0x7fea440c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 --2- 172.21.15.49:0/2321869220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fea44157530 0x7fea440c9cf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fea40011ad0 tx=0x7fea40011f90 comp rx=0 tx=0).stop 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea490ba640 1 -- 172.21.15.49:0/2321869220 reap_dead start 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 shutdown_connections 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.295+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 >> 172.21.15.49:0/2321869220 conn(0x7fea44074840 msgr2=0x7fea440cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.296+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 shutdown_connections 2024-05-09T08:58:30.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.296+0000 7fea4ab1d640 1 -- 172.21.15.49:0/2321869220 wait complete. 2024-05-09T08:58:30.367 INFO:teuthology.orchestra.run.smithi049.stdout:279172874245 2024-05-09T08:58:30.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:58:30.368 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:30.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:58:30.369 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:30.369 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:58:30.370 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:30.370 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:58:30.371 INFO:teuthology.orchestra.run.smithi049.stdout:55834574884 2024-05-09T08:58:30.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:58:30.372 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:30.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:58:30.462 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.463+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.464+0000 7fe770b72640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.468 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.465+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe74c076f50 con 0x7fe74c0743e0 2024-05-09T08:58:30.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.496+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe750002dd0 con 0x7fe74c0743e0 2024-05-09T08:58:30.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.497+0000 7fe768ff9640 1 -- 172.21.15.49:0/184075524 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe750002dd0 con 0x7fe74c0743e0 2024-05-09T08:58:30.496 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 msgr2=0x7fe74c076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fe74c0743e0 0x7fe74c076880 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe74c06bf00 msgr2=0x7fe74c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe74c06bf00 0x7fe74c06e3c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe764032000 tx=0x7fe764042040 comp rx=0 tx=0).stop 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 msgr2=0x7fe76c1ee410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 --2- 172.21.15.49:0/184075524 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe76c0c8e60 0x7fe76c1ee410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe754011830 tx=0x7fe754011cf0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe770b72640 1 -- 172.21.15.49:0/184075524 reap_dead start 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 shutdown_connections 2024-05-09T08:58:30.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.498+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 >> 172.21.15.49:0/184075524 conn(0x7fe76c0cc290 msgr2=0x7fe76c066ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.499+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 shutdown_connections 2024-05-09T08:58:30.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.499+0000 7fe7725d5640 1 -- 172.21.15.49:0/184075524 wait complete. 2024-05-09T08:58:30.515 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:30.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:58:30.555 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.556+0000 7f38ca929640 1 Processor -- start 2024-05-09T08:58:30.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.556+0000 7f38ca929640 1 -- start start 2024-05-09T08:58:30.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.557+0000 7f38ca929640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38c4155190 0x7f38c40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.557+0000 7f38ca929640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 0x7f38c40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.557+0000 7f38ca929640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c415e410 con 0x7f38c40b3530 2024-05-09T08:58:30.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.557+0000 7f38ca929640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f38c415ae60 con 0x7f38c4155190 2024-05-09T08:58:30.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.557+0000 7f38ca929640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38c415b120 con 0x7f38c40cba60 2024-05-09T08:58:30.558 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.558+0000 7f38c8ec6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 0x7f38c40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.558+0000 7f38c37fe640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f38c40b3530 0x7f38c40cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49104/0 (socket says 172.21.15.49:49104) 2024-05-09T08:58:30.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.558+0000 7f38c37fe640 1 -- 172.21.15.49:0/1790240129 learned_addr learned my addr 172.21.15.49:0/1790240129 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.558+0000 7f38c3fff640 1 --2- 172.21.15.49:0/1790240129 >> v2:172.21.15.183:3300/0 conn(0x7f38c4155190 0x7f38c40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.559+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249592967 0 0) 0x7f38c415e410 con 0x7f38c40b3530 2024-05-09T08:58:30.559 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.559+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b0003520 con 0x7f38c40b3530 2024-05-09T08:58:30.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 871604009 0 0) 0x7f38b0003520 con 0x7f38c40b3530 2024-05-09T08:58:30.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 msgr2=0x7f38c40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.562 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 0x7f38c40cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 >> v2:172.21.15.183:3300/0 conn(0x7f38c4155190 msgr2=0x7f38c40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 --2- 172.21.15.49:0/1790240129 >> v2:172.21.15.183:3300/0 conn(0x7f38c4155190 0x7f38c40cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c415b400 con 0x7f38c40b3530 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f38ac001690 con 0x7f38c40b3530 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 >> v1:172.21.15.183:6789/0 conn(0x7f38c40b3530 legacy=0x7f38c40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.561+0000 7f38c2ffd640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.562+0000 7f38c2ffd640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38b0004800 0x7f38b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.562+0000 7f38c2ffd640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 0x7f38b00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.562+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38c415ae60 con 0x7f38b0004800 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.562+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38c415b120 con 0x7f38b00039b0 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.562+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38b000bc10 con 0x7f38c40cba60 2024-05-09T08:58:30.563 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38b0004800 0x7f38b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c3fff640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.564 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 -- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 msgr2=0x7f38b00051e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c40cba60 0x7f38b00051e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 -- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b00039b0 msgr2=0x7f38b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c37fe640 1 -- 172.21.15.49:0/1790240129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c415b400 con 0x7f38b0004800 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.563+0000 7f38c3fff640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b00039b0 0x7f38b0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.565+0000 7f38c37fe640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38b0004800 0x7f38b0004c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c260 tx=0x7f38b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4940bf1d4fcb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.565+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac007d80 con 0x7f38b0004800 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.565+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac02e070 con 0x7f38b0004800 2024-05-09T08:58:30.565 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.565+0000 7f38c2ffd640 1 -- 172.21.15.49:0/1790240129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac005d50 con 0x7f38b0004800 2024-05-09T08:58:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.566+0000 7f38ca929640 1 -- 172.21.15.49:0/1790240129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38b0004800 msgr2=0x7f38b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.567+0000 7f38ca929640 1 --2- 172.21.15.49:0/1790240129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38b0004800 0x7f38b0004c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c260 tx=0x7f38b0005830 comp rx=0 tx=0).stop 2024-05-09T08:58:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.567+0000 7f38c8ec6640 1 -- 172.21.15.49:0/1790240129 reap_dead start 2024-05-09T08:58:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.567+0000 7f38ca929640 1 -- 172.21.15.49:0/1790240129 shutdown_connections 2024-05-09T08:58:30.566 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.567+0000 7f38ca929640 1 -- 172.21.15.49:0/1790240129 >> 172.21.15.49:0/1790240129 conn(0x7f38c4074a20 msgr2=0x7f38c4074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.567 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.568+0000 7f38ca929640 1 -- 172.21.15.49:0/1790240129 shutdown_connections 2024-05-09T08:58:30.568 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.568+0000 7f38ca929640 1 -- 172.21.15.49:0/1790240129 wait complete. 2024-05-09T08:58:30.569 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.571+0000 7f38ca929640 1 Processor -- start 2024-05-09T08:58:30.570 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.571+0000 7f38ca929640 1 -- start start 2024-05-09T08:58:30.571 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.572+0000 7f38ca929640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.573+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.573+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45528/0 (socket says 172.21.15.49:45528) 2024-05-09T08:58:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.573+0000 7f38c3fff640 1 -- 172.21.15.49:0/408635208 learned_addr learned my addr 172.21.15.49:0/408635208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.572 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.573+0000 7f38ca929640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 0x7f38c40b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.573+0000 7f38ca929640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 0x7f38c414f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.574+0000 7f38c37fe640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 0x7f38c40b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.573 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.574+0000 7f38c8ec6640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 0x7f38c414f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.574+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38c415bca0 con 0x7f38c40b5fd0 2024-05-09T08:58:30.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.574+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38c415b9c0 con 0x7f38c414f430 2024-05-09T08:58:30.574 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.574+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38c4155570 con 0x7f38c4154580 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 -- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 msgr2=0x7f38c40b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 0x7f38c40b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 -- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 msgr2=0x7f38c414f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 0x7f38c414f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c41f1050 con 0x7f38c40b5fd0 2024-05-09T08:58:30.575 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.575+0000 7f38c3fff640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f38b400f8b0 tx=0x7f38b400fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.576+0000 7f38c37fe640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 0x7f38c40b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.576+0000 7f38c8ec6640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 0x7f38c414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.576+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4019020 con 0x7f38c40b5fd0 2024-05-09T08:58:30.576 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.576+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41549e0 con 0x7f38c40b5fd0 2024-05-09T08:58:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.576+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41f12e0 con 0x7f38c40b5fd0 2024-05-09T08:58:30.577 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.578+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b400d8e0 con 0x7f38c40b5fd0 2024-05-09T08:58:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.578+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400dcb0 con 0x7f38c40b5fd0 2024-05-09T08:58:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.579+0000 7f29e0dff640 1 Processor -- start 2024-05-09T08:58:30.578 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.579+0000 7f29e0dff640 1 -- start start 2024-05-09T08:58:30.579 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.579+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c404f660 con 0x7f38c40b5fd0 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38b401c020 con 0x7f38c40b5fd0 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f29e0dff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f29d40d7010 0x7f29d4036620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f29e0dff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d4006220 0x7f29d40e0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f29e0dff640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d4036b60 con 0x7f29d40e0d50 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f29e0dff640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f29d404b3b0 con 0x7f29d40d7010 2024-05-09T08:58:30.580 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f29e0dff640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f29d40345d0 con 0x7f29d4006220 2024-05-09T08:58:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.581+0000 7f38c0ff9640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 0x7f38a406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.582+0000 7f38c37fe640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 0x7f38a406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.581 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.582+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f38b408b690 con 0x7f38c40b5fd0 2024-05-09T08:58:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.582+0000 7fd94ccd6640 1 Processor -- start 2024-05-09T08:58:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.582+0000 7fd94ccd6640 1 -- start start 2024-05-09T08:58:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.583+0000 7f38c37fe640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 0x7f38a406e3f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38ac033000 tx=0x7f38ac009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.583 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94ccd6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 0x7fd948151000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94ccd6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd94806dac0 0x7fd94806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94ccd6640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd948151540 con 0x7fd9480c7910 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94ccd6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fd948157d20 con 0x7fd9480ccdb0 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94ccd6640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fd9481571a0 con 0x7fd94806dac0 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7f29db7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f29d40d7010 0x7f29d4036620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7f29db7fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f29d40d7010 0x7f29d4036620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45540/0 (socket says 172.21.15.49:45540) 2024-05-09T08:58:30.584 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7f29db7fe640 1 -- 172.21.15.49:0/2079465610 learned_addr learned my addr 172.21.15.49:0/2079465610 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 0x7fd948151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.584+0000 7fd94659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 0x7fd948151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45556/0 (socket says 172.21.15.49:45556) 2024-05-09T08:58:30.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd94659c640 1 -- 172.21.15.49:0/2149693308 learned_addr learned my addr 172.21.15.49:0/2149693308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd946d9d640 1 --2- 172.21.15.49:0/2149693308 >> v2:172.21.15.183:3300/0 conn(0x7fd94806dac0 0x7fd94806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.585 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7f29db7fe640 1 -- 172.21.15.49:0/2079465610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d4006220 msgr2=0x7f29d40e0810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:58:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7f29db7fe640 1 --2- 172.21.15.49:0/2079465610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d4006220 0x7f29d40e0810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7f29db7fe640 1 -- 172.21.15.49:0/2079465610 >> v1:172.21.15.49:6789/0 conn(0x7f29d40e0d50 legacy=0x7f29d4006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7f29db7fe640 1 -- 172.21.15.49:0/2079465610 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d40348b0 con 0x7f29d40d7010 2024-05-09T08:58:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd946d9d640 1 -- 172.21.15.49:0/2149693308 >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 msgr2=0x7fd948151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd946d9d640 1 --2- 172.21.15.49:0/2149693308 >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 0x7fd948151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd946d9d640 1 -- 172.21.15.49:0/2149693308 >> v1:172.21.15.49:6789/0 conn(0x7fd9480c7910 legacy=0x7fd9480c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.585+0000 7fd946d9d640 1 -- 172.21.15.49:0/2149693308 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd948157480 con 0x7fd94806dac0 2024-05-09T08:58:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.586+0000 7f29db7fe640 1 --2- 172.21.15.49:0/2079465610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d40d7010 0x7f29d4036620 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29dc065110 tx=0x7f29dc0a6c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8505c8746968ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.586+0000 7fd946d9d640 1 --2- 172.21.15.49:0/2149693308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94806dac0 0x7fd94806df30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd948071f40 tx=0x7fd94002a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba87ba0f5f748381 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.586+0000 7f18c51b7640 1 Processor -- start 2024-05-09T08:58:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.586+0000 7f18c51b7640 1 -- start start 2024-05-09T08:58:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.587+0000 7fd94559a640 1 -- 172.21.15.49:0/2149693308 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940007540 con 0x7fd94806dac0 2024-05-09T08:58:30.588 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.587+0000 7f35f6c2a640 1 Processor -- start 2024-05-09T08:58:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.587+0000 7fd94559a640 1 -- 172.21.15.49:0/2149693308 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9400076e0 con 0x7fd94806dac0 2024-05-09T08:58:30.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.587+0000 7f35f6c2a640 1 -- start start 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.587+0000 7fd94559a640 1 -- 172.21.15.49:0/2149693308 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940007ab0 con 0x7fd94806dac0 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 -- 172.21.15.49:0/2149693308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94806dac0 msgr2=0x7fd94806df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/2149693308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94806dac0 0x7fd94806df30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd948071f40 tx=0x7fd94002a620 comp rx=0 tx=0).stop 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18c51b7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 0x7f18c00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18c51b7640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c015e410 con 0x7f18c0155190 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18c51b7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c015ae60 con 0x7f18c00cb890 2024-05-09T08:58:30.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18c51b7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f18c015b120 con 0x7f18c00b3530 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 -- 172.21.15.49:0/2149693308 shutdown_connections 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/2149693308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94806dac0 0x7fd94806df30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/2149693308 >> v2:172.21.15.49:3300/0 conn(0x7fd9480ccdb0 0x7fd948151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 -- 172.21.15.49:0/2149693308 >> 172.21.15.49:0/2149693308 conn(0x7fd9480cc290 msgr2=0x7fd9480cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 -- 172.21.15.49:0/2149693308 shutdown_connections 2024-05-09T08:58:30.591 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f5705cb3640 1 Processor -- start 2024-05-09T08:58:30.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f6c2a640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 0x7f35f015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f5705cb3640 1 -- start start 2024-05-09T08:58:30.592 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18be59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 0x7f18c00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18bf59e640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f18c00cb890 0x7f18c015e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49120/0 (socket says 172.21.15.49:49120) 2024-05-09T08:58:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f18bf59e640 1 -- 172.21.15.49:0/2329413889 learned_addr learned my addr 172.21.15.49:0/2329413889 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f6c2a640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 0x7f35f00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.593 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f7a8dea5640 1 Processor -- start 2024-05-09T08:58:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f5c28640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 0x7f35f015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f5c28640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 0x7f35f015f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49108/0 (socket says 172.21.15.49:49108) 2024-05-09T08:58:30.594 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f5c28640 1 -- 172.21.15.49:0/808087291 learned_addr learned my addr 172.21.15.49:0/808087291 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f18be59c640 1 -- 172.21.15.49:0/2329413889 >> v1:172.21.15.183:6789/0 conn(0x7f18c00cb890 legacy=0x7f18c015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f18be59c640 1 -- 172.21.15.49:0/2329413889 >> v1:172.21.15.49:6789/0 conn(0x7f18c0155190 legacy=0x7f18c00cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.595 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f0163480 con 0x7f35f00c9c40 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f5427640 1 --2- 172.21.15.49:0/808087291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 0x7f35f00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f35f006f4e0 con 0x7f35f0158f40 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f35f0162ff0 con 0x7f35f00731d0 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f4c26640 1 -- 172.21.15.49:0/808087291 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929310434 0 0) 0x7f35f0163480 con 0x7f35f00c9c40 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f4c26640 1 -- 172.21.15.49:0/808087291 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35dc003520 con 0x7f35f00c9c40 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f5427640 1 -- 172.21.15.49:0/808087291 >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 msgr2=0x7f35f015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f5427640 1 --2- 172.21.15.49:0/808087291 >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 0x7f35f015f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f5427640 1 -- 172.21.15.49:0/808087291 >> v1:172.21.15.183:6789/0 conn(0x7f35f00c9c40 legacy=0x7f35f0071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.596 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f35f5427640 1 -- 172.21.15.49:0/808087291 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f006f830 con 0x7f35f00731d0 2024-05-09T08:58:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f5705cb3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f56f80dd3c0 0x7f56f8046460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f5705cb3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.589+0000 7f18be59c640 1 -- 172.21.15.49:0/2329413889 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c015b400 con 0x7f18c00b3530 2024-05-09T08:58:30.597 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f5705cb3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f56f80e3090 0x7f56f80e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7f7a8dea5640 1 -- start start 2024-05-09T08:58:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f56feffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a88150fb0 0x7f7a880c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 0x7f7a880c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.598 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a880c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8815a430 con 0x7f7a880c7420 2024-05-09T08:58:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8815a710 con 0x7f7a88150fb0 2024-05-09T08:58:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.588+0000 7fd94ccd6640 1 -- 172.21.15.49:0/2149693308 wait complete. 2024-05-09T08:58:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a8dea5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8815a9f0 con 0x7f7a880c7c50 2024-05-09T08:58:30.599 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7fd94ccd6640 1 Processor -- start 2024-05-09T08:58:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f56feffd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47634/0 (socket says 172.21.15.49:47634) 2024-05-09T08:58:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f56feffd640 1 -- 172.21.15.49:0/1315318890 learned_addr learned my addr 172.21.15.49:0/1315318890 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f56fffff640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.49:3300/0 conn(0x7f56f80e3090 0x7f56f80e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.600 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f56f8045450 con 0x7f56f80e3090 2024-05-09T08:58:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f18be59c640 1 --2- 172.21.15.49:0/2329413889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 0x7f18c00cb350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0097a0 tx=0x7f18ac02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7315ebe97c49dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f56f8045fd0 con 0x7f56f80dd3c0 2024-05-09T08:58:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f56f80dd7a0 con 0x7f56f804ba60 2024-05-09T08:58:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a877fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a88150fb0 0x7f7a880c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.601 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a877fe640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a88150fb0 0x7f7a880c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49124/0 (socket says 172.21.15.49:49124) 2024-05-09T08:58:30.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f7a877fe640 1 -- 172.21.15.49:0/1499211268 learned_addr learned my addr 172.21.15.49:0/1499211268 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7fd94ccd6640 1 -- start start 2024-05-09T08:58:30.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f56ff7fe640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.183:3300/0 conn(0x7f56f80dd3c0 0x7f56f8046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.602 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f35f5427640 1 --2- 172.21.15.49:0/808087291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 0x7f35f00c9700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35e00097a0 tx=0x7f35e002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f281ef2e971a9a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.590+0000 7f18bdd9b640 1 -- 172.21.15.49:0/2329413889 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037070 con 0x7f18c00b3530 2024-05-09T08:58:30.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a877fe640 1 -- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 msgr2=0x7f7a880c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.603 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f35f4c26640 1 -- 172.21.15.49:0/808087291 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0037070 con 0x7f35f00731d0 2024-05-09T08:58:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38b4023020 con 0x7f38c40b5fd0 2024-05-09T08:58:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f35f4c26640 1 -- 172.21.15.49:0/808087291 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e0032440 con 0x7f35f00731d0 2024-05-09T08:58:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f35f4c26640 1 -- 172.21.15.49:0/808087291 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0032810 con 0x7f35f00731d0 2024-05-09T08:58:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a86ffd640 1 --2- 172.21.15.49:0/1499211268 >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 0x7f7a880c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a87fff640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a880c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f29da7fc640 1 -- 172.21.15.49:0/2079465610 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0b4070 con 0x7f29d40d7010 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f56feffd640 1 -- 172.21.15.49:0/1315318890 >> v2:172.21.15.183:3300/0 conn(0x7f56f80dd3c0 msgr2=0x7f56f8046460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f56feffd640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.183:3300/0 conn(0x7f56f80dd3c0 0x7f56f8046460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f56feffd640 1 -- 172.21.15.49:0/1315318890 >> v2:172.21.15.49:3300/0 conn(0x7f56f80e3090 msgr2=0x7f56f80e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f56feffd640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.49:3300/0 conn(0x7f56f80e3090 0x7f56f80e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.605 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f56feffd640 1 -- 172.21.15.49:0/1315318890 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f80e3e90 con 0x7f56f804ba60 2024-05-09T08:58:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a877fe640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a880c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a877fe640 1 -- 172.21.15.49:0/1499211268 >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 msgr2=0x7f7a880c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a877fe640 1 --2- 172.21.15.49:0/1499211268 >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 0x7f7a880c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f29da7fc640 1 -- 172.21.15.49:0/2079465610 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc0a7ab0 con 0x7f29d40d7010 2024-05-09T08:58:30.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a877fe640 1 -- 172.21.15.49:0/1499211268 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8815acd0 con 0x7f7a88150fb0 2024-05-09T08:58:30.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f29da7fc640 1 -- 172.21.15.49:0/2079465610 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0af440 con 0x7f29d40d7010 2024-05-09T08:58:30.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f18bdd9b640 1 -- 172.21.15.49:0/2329413889 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac032440 con 0x7f18c00b3530 2024-05-09T08:58:30.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f18bdd9b640 1 -- 172.21.15.49:0/2329413889 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac032810 con 0x7f18c00b3530 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7f7a86ffd640 1 --2- 172.21.15.49:0/1499211268 >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 0x7f7a880c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9480c7910 0x7fd94813c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9480ccdb0 0x7fd94813e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd94815a420 con 0x7fd94806dac0 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd948157a40 con 0x7fd9480ccdb0 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.591+0000 7fd94ccd6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd948156dd0 con 0x7fd9480c7910 2024-05-09T08:58:30.608 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f7a87fff640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a880c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7fd94659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 -- 172.21.15.49:0/2329413889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 msgr2=0x7f18c00cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2329413889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 0x7f18c00cb350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0097a0 tx=0x7f18ac02a250 comp rx=0 tx=0).stop 2024-05-09T08:58:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 msgr2=0x7f35f00c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/808087291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 0x7f35f00c9700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35e00097a0 tx=0x7f35e002a230 comp rx=0 tx=0).stop 2024-05-09T08:58:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7fd94659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45596/0 (socket says 172.21.15.49:45596) 2024-05-09T08:58:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7fd94659c640 1 -- 172.21.15.49:0/3281024492 learned_addr learned my addr 172.21.15.49:0/3281024492 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 shutdown_connections 2024-05-09T08:58:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/808087291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f00731d0 0x7f35f00c9700 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35e00097a0 tx=0x7f35e002a230 comp rx=0 tx=0).stop 2024-05-09T08:58:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/808087291 >> v2:172.21.15.183:3300/0 conn(0x7f35f0158f40 0x7f35f015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 >> 172.21.15.49:0/808087291 conn(0x7f35f006b200 msgr2=0x7f35f006b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 -- 172.21.15.49:0/2329413889 shutdown_connections 2024-05-09T08:58:30.611 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2329413889 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00b3530 0x7f18c00cb350 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0097a0 tx=0x7f18ac02a250 comp rx=0 tx=0).stop 2024-05-09T08:58:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 shutdown_connections 2024-05-09T08:58:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f35f6c2a640 1 -- 172.21.15.49:0/808087291 wait complete. 2024-05-09T08:58:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 -- 172.21.15.49:0/2329413889 >> 172.21.15.49:0/2329413889 conn(0x7f18c0074a20 msgr2=0x7f18c0074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f56feffd640 1 --2- 172.21.15.49:0/1315318890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f56f80079c0 tx=0x7f56ec02a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cea21626ac4d659 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7fd945d9b640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9480c7910 0x7fd94813c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7fd946d9d640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9480ccdb0 0x7fd94813e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 -- 172.21.15.49:0/2329413889 shutdown_connections 2024-05-09T08:58:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f18c51b7640 1 -- 172.21.15.49:0/2329413889 wait complete. 2024-05-09T08:58:30.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.592+0000 7f29e0dff640 1 -- 172.21.15.49:0/2079465610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d40d7010 msgr2=0x7f29d4036620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 -- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9480c7910 msgr2=0x7fd94813c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f29e0dff640 1 --2- 172.21.15.49:0/2079465610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d40d7010 0x7f29d4036620 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f29dc065110 tx=0x7f29dc0a6c60 comp rx=0 tx=0).stop 2024-05-09T08:58:30.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f56fe7fc640 1 -- 172.21.15.49:0/1315318890 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec037070 con 0x7f56f804ba60 2024-05-09T08:58:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9480c7910 0x7fd94813c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f56fe7fc640 1 -- 172.21.15.49:0/1315318890 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ec032440 con 0x7f56f804ba60 2024-05-09T08:58:30.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f56fe7fc640 1 -- 172.21.15.49:0/1315318890 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec032810 con 0x7f56f804ba60 2024-05-09T08:58:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 -- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9480ccdb0 msgr2=0x7fd94813e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9480ccdb0 0x7fd94813e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 -- 172.21.15.49:0/3281024492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94813fd50 con 0x7fd94806dac0 2024-05-09T08:58:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f29e0dff640 1 -- 172.21.15.49:0/2079465610 shutdown_connections 2024-05-09T08:58:30.616 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f29e0dff640 1 --2- 172.21.15.49:0/2079465610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d4006220 0x7f29d40e0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f7a877fe640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88150fb0 0x7f7a880c6ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a880736f0 tx=0x7f7a8002bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51227f901b40e972 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f29e0dff640 1 --2- 172.21.15.49:0/2079465610 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d40d7010 0x7f29d4036620 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f29e0dff640 1 -- 172.21.15.49:0/2079465610 >> 172.21.15.49:0/2079465610 conn(0x7f29d404d4b0 msgr2=0x7f29d404d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7fd94659c640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e8b0 tx=0x7fd93c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c017070 con 0x7fd94806dac0 2024-05-09T08:58:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f7a867fc640 1 -- 172.21.15.49:0/1499211268 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80033070 con 0x7f7a88150fb0 2024-05-09T08:58:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f7a867fc640 1 -- 172.21.15.49:0/1499211268 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a800094e0 con 0x7f7a88150fb0 2024-05-09T08:58:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.593+0000 7f7a867fc640 1 -- 172.21.15.49:0/1499211268 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800098b0 con 0x7f7a88150fb0 2024-05-09T08:58:30.618 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9480cb960 con 0x7fd94806dac0 2024-05-09T08:58:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd948072370 con 0x7fd94806dac0 2024-05-09T08:58:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f35f6c2a640 1 Processor -- start 2024-05-09T08:58:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c00c940 con 0x7fd94806dac0 2024-05-09T08:58:30.619 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00cd10 con 0x7fd94806dac0 2024-05-09T08:58:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 msgr2=0x7f56f804be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f5705cb3640 1 --2- 172.21.15.49:0/1315318890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f56f80079c0 tx=0x7f56ec02a1d0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f29e0dff640 1 -- 172.21.15.49:0/2079465610 shutdown_connections 2024-05-09T08:58:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f29e0dff640 1 -- 172.21.15.49:0/2079465610 wait complete. 2024-05-09T08:58:30.620 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f18c51b7640 1 Processor -- start 2024-05-09T08:58:30.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.594+0000 7f35f6c2a640 1 -- start start 2024-05-09T08:58:30.621 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.595+0000 7f35f6c2a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f5c28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f5c28640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45608/0 (socket says 172.21.15.49:45608) 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f5c28640 1 -- 172.21.15.49:0/166040599 learned_addr learned my addr 172.21.15.49:0/166040599 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 0x7f35f01f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f015fa80 0x7f35f00c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.595+0000 7f7a8dea5640 1 -- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88150fb0 msgr2=0x7f7a880c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.595+0000 7f7a8dea5640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88150fb0 0x7f7a880c6ee0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a880736f0 tx=0x7f7a8002bae0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 -- 172.21.15.49:0/1499211268 shutdown_connections 2024-05-09T08:58:30.622 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a880c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 --2- 172.21.15.49:0/1499211268 >> v2:172.21.15.49:3300/0 conn(0x7f7a880c7420 0x7f7a880c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 --2- 172.21.15.49:0/1499211268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88150fb0 0x7f7a880c6ee0 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7a880736f0 tx=0x7f7a8002bae0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 -- 172.21.15.49:0/1499211268 >> 172.21.15.49:0/1499211268 conn(0x7f7a88074840 msgr2=0x7f7a88074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f29e0dff640 1 Processor -- start 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 shutdown_connections 2024-05-09T08:58:30.623 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 -- 172.21.15.49:0/1499211268 shutdown_connections 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f29e0dff640 1 -- start start 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.595+0000 7f18c51b7640 1 -- start start 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00cb890 0x7f18c01441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 0x7f18c0154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f18c015bca0 con 0x7f18c015c9f0 2024-05-09T08:58:30.624 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18c015b9c0 con 0x7f18c0155190 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18c51b7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f18c0155570 con 0x7f18c00cb890 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f5705cb3640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.49:3300/0 conn(0x7f56f80e3090 0x7f56f80e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f5705cb3640 1 --2- 172.21.15.49:0/1315318890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f804be80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f5705cb3640 1 --2- 172.21.15.49:0/1315318890 >> v2:172.21.15.183:3300/0 conn(0x7f56f80dd3c0 0x7f56f8046460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 >> 172.21.15.49:0/1315318890 conn(0x7f56f8050180 msgr2=0x7f56f8050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.625 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49170/0 (socket says 172.21.15.49:49170) 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 -- 172.21.15.49:0/2090201843 learned_addr learned my addr 172.21.15.49:0/2090201843 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0162700 con 0x7f35f00c9c40 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35f006ef80 con 0x7f35f0158f40 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f35f01629a0 con 0x7f35f015fa80 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f35f6429640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f015fa80 0x7f35f00c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5427640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 0x7f35f01f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5c28640 1 -- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f015fa80 msgr2=0x7f35f00c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5c28640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f015fa80 0x7f35f00c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5c28640 1 -- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 msgr2=0x7f35f01f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.626 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5c28640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 0x7f35f01f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5c28640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f01f8f80 con 0x7f35f00c9c40 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f35f5427640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 0x7f35f01f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c005290 con 0x7fd94806dac0 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18bf59e640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 0x7f18c0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18bed9d640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00cb890 0x7f18c01441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 -- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00cb890 msgr2=0x7f18c01441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00cb890 0x7f18c01441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 -- 172.21.15.49:0/2090201843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 msgr2=0x7f18c0154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd93c01a070 con 0x7fd94806dac0 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 0x7f18c0154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18be59c640 1 -- 172.21.15.49:0/2090201843 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c015dd30 con 0x7f18c0155190 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.597+0000 7f18bf59e640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 0x7f18c0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:30.628 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f35f5c28640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ebe0 tx=0x7f35ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18be59c640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0098c0 tx=0x7f18ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.629 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d40d7010 0x7f29d4048dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29db7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45622/0 (socket says 172.21.15.49:45622) 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29db7fe640 1 -- 172.21.15.49:0/3162967846 learned_addr learned my addr 172.21.15.49:0/3162967846 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d40e0d50 0x7f29d4049300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29d40e02f0 con 0x7f29d4006220 2024-05-09T08:58:30.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29d4034e70 con 0x7f29d40e0d50 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac041070 con 0x7f18c0155190 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c015dfd0 con 0x7f18c0155190 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01f5920 con 0x7f18c0155190 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac03c870 con 0x7f18c0155190 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037040 con 0x7f18c0155190 2024-05-09T08:58:30.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f29d40075d0 con 0x7f29d40d7010 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7fd936ffd640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 0x7fd92806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 -- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d40d7010 msgr2=0x7f29d4048dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d40d7010 0x7f29d4048dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 -- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d40e0d50 msgr2=0x7f29d4049300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d40e0d50 0x7f29d4049300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d404a8f0 con 0x7f29d4006220 2024-05-09T08:58:30.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 shutdown_connections 2024-05-09T08:58:30.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f5705cb3640 1 -- 172.21.15.49:0/1315318890 wait complete. 2024-05-09T08:58:30.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7fd945d9b640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 0x7fd92806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f29db7fe640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29dc050860 tx=0x7f29dc0692e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fd93c08b1b0 con 0x7fd94806dac0 2024-05-09T08:58:30.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.596+0000 7f7a8dea5640 1 -- 172.21.15.49:0/1499211268 wait complete. 2024-05-09T08:58:30.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.598+0000 7f7a8dea5640 1 Processor -- start 2024-05-09T08:58:30.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c01f1620 con 0x7f18c0155190 2024-05-09T08:58:30.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.599+0000 7f7a8dea5640 1 -- start start 2024-05-09T08:58:30.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7fd945d9b640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 0x7fd92806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd948154970 tx=0x7fd938009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f5705cb3640 1 Processor -- start 2024-05-09T08:58:30.635 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18ac044070 con 0x7f18c0155190 2024-05-09T08:58:30.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a88154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88159430 0x7f7a881425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.636 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f5705cb3640 1 -- start start 2024-05-09T08:58:30.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45624/0 (socket says 172.21.15.49:45624) 2024-05-09T08:58:30.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a877fe640 1 -- 172.21.15.49:0/3627348228 learned_addr learned my addr 172.21.15.49:0/3627348228 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88068560 con 0x7f7a880c7420 2024-05-09T08:58:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88151390 con 0x7f7a88159430 2024-05-09T08:58:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a86ffd640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a88154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.601+0000 7f7a8dea5640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7a880686e0 con 0x7f7a880c7c50 2024-05-09T08:58:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f18b77fe640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 0x7f18a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a87fff640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88159430 0x7f7a881425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f18ac0aba00 con 0x7f18c0155190 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 -- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 msgr2=0x7f7a88154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a88154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 -- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88159430 msgr2=0x7f7a881425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88159430 0x7f7a881425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881ec5d0 con 0x7f7a880c7420 2024-05-09T08:58:30.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f18bed9d640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 0x7f18a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.602+0000 7f7a877fe640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002c0d0 tx=0x7f7a80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f80dd3c0 0x7f56f8006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56f80e3090 0x7f56f80cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8045c60 con 0x7f56f80dd3c0 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8045980 con 0x7f56f80e3090 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f5705cb3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f56f8005e90 con 0x7f56f804ba60 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47700/0 (socket says 172.21.15.49:47700) 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 -- 172.21.15.49:0/716365416 learned_addr learned my addr 172.21.15.49:0/716365416 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56feffd640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f80dd3c0 0x7f56f8006ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.641 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56fffff640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56f80e3090 0x7f56f80cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f18bed9d640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 0x7f18a806e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007fd0 tx=0x7f18b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 -- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56f80e3090 msgr2=0x7f56f80cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56f80e3090 0x7f56f80cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 -- 172.21.15.49:0/716365416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f80dd3c0 msgr2=0x7f56f8006ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f80dd3c0 0x7f56f8006ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.603+0000 7f56ff7fe640 1 -- 172.21.15.49:0/716365416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f804fe80 con 0x7f56f804ba60 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f56ff7fe640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f56f400d800 tx=0x7f56f400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4018020 con 0x7f56f804ba60 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56f4009640 con 0x7f56f804ba60 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f80d9f10 con 0x7f56f804ba60 2024-05-09T08:58:30.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4009a10 con 0x7f56f804ba60 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.604+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f80da4d0 con 0x7f56f804ba60 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.605+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0bf020 con 0x7f29d4006220 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.605+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d40db970 con 0x7f29d4006220 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.605+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc0ba730 con 0x7f29d4006220 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.605+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0b4040 con 0x7f29d4006220 2024-05-09T08:58:30.644 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.605+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d40dbc30 con 0x7f29d4006220 2024-05-09T08:58:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f8005390 con 0x7f56f804ba60 2024-05-09T08:58:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd93c0593c0 con 0x7fd94806dac0 2024-05-09T08:58:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f56f401b020 con 0x7f56f804ba60 2024-05-09T08:58:30.645 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80047070 con 0x7f7a880c7420 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f7a8dea5640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a88159d60 con 0x7f7a880c7420 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec017070 con 0x7f35f00c9c40 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec00ed90 con 0x7f35f00c9c40 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec00ce50 con 0x7f35f00c9c40 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.606+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f01f4f60 con 0x7f35f00c9c40 2024-05-09T08:58:30.646 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01f5220 con 0x7f35f00c9c40 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8000d050 con 0x7f7a880c7420 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f7a8dea5640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881ec890 con 0x7f7a880c7420 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc005290 con 0x7f29d4006220 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f56ebfff640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 0x7f56e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.607+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f56f408b040 con 0x7f56f804ba60 2024-05-09T08:58:30.647 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.608+0000 7f56feffd640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 0x7f56e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.608+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80033040 con 0x7f7a880c7420 2024-05-09T08:58:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.610+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f18c01f1620 con 0x7f18c0155190 2024-05-09T08:58:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.609+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29dc0c2020 con 0x7f29d4006220 2024-05-09T08:58:30.648 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.609+0000 7f29c7fff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.610+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f29dc1288d0 con 0x7f29d4006220 2024-05-09T08:58:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.610+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8804f5f0 con 0x7f7a880c7420 2024-05-09T08:58:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a8004a070 con 0x7f7a880c7420 2024-05-09T08:58:30.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f29daffd640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.610+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35ec01a070 con 0x7f35f00c9c40 2024-05-09T08:58:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f35e67fc640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 0x7f35d406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f35ec08ad20 con 0x7f35f00c9c40 2024-05-09T08:58:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f56feffd640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 0x7f56e406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56ec034000 tx=0x7f56ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8005290 con 0x7f35f00c9c40 2024-05-09T08:58:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.611+0000 7f7a6ffff640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 0x7f7a7406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f35f5427640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 0x7f35d406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f7a800abe80 con 0x7f7a880c7420 2024-05-09T08:58:30.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f7a86ffd640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 0x7f7a7406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f29daffd640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 0x7f29c006e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f29d4035730 tx=0x7f29c8006b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56f4022030 con 0x7f56f804ba60 2024-05-09T08:58:30.652 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f7a86ffd640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 0x7f7a7406e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a88142e90 tx=0x7f7a78008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.612+0000 7f35f5427640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 0x7f35d406e3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f35e0034000 tx=0x7f35e002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.619+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f29dc0c9030 con 0x7f29d4006220 2024-05-09T08:58:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.629+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7a8007a110 con 0x7f7a880c7420 2024-05-09T08:58:30.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.638+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35ec059040 con 0x7f35f00c9c40 2024-05-09T08:58:30.744 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.740+0000 7f3a1759e640 1 Processor -- start 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.740+0000 7f3a1759e640 1 -- start start 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d5de0 0x7f3a1004bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3a1004c170 0x7f3a1004ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a1004d800 0x7f3a1004cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3a100df120 con 0x7f3a1004d800 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3a100dbb30 con 0x7f3a1004c170 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a1759e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100dbdf0 con 0x7f3a100d5de0 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a1004d800 0x7f3a1004cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3a1004d800 0x7f3a1004cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45644/0 (socket says 172.21.15.49:45644) 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 -- 172.21.15.49:0/2185452571 learned_addr learned my addr 172.21.15.49:0/2185452571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 -- 172.21.15.49:0/2185452571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d5de0 msgr2=0x7f3a1004bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 --2- 172.21.15.49:0/2185452571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d5de0 0x7f3a1004bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 -- 172.21.15.49:0/2185452571 >> v2:172.21.15.183:3300/0 conn(0x7f3a1004c170 msgr2=0x7f3a1004ca90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:58:30.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 --2- 172.21.15.49:0/2185452571 >> v2:172.21.15.183:3300/0 conn(0x7f3a1004c170 0x7f3a1004ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.742+0000 7f3a16d9d640 1 -- 172.21.15.49:0/2185452571 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a100dc0d0 con 0x7f3a1004d800 2024-05-09T08:58:30.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.743+0000 7f3a16d9d640 1 --2- 172.21.15.49:0/2185452571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004d800 0x7f3a1004cfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a18065110 tx=0x7f3a180a6cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a21c4797578797 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.745+0000 7f3a1559a640 1 -- 172.21.15.49:0/2185452571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a180b4070 con 0x7f3a1004d800 2024-05-09T08:58:30.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.745+0000 7f3a1559a640 1 -- 172.21.15.49:0/2185452571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a180a7ac0 con 0x7f3a1004d800 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 -- 172.21.15.49:0/2185452571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004d800 msgr2=0x7f3a1004cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 --2- 172.21.15.49:0/2185452571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004d800 0x7f3a1004cfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a18065110 tx=0x7f3a180a6cb0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 -- 172.21.15.49:0/2185452571 shutdown_connections 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 --2- 172.21.15.49:0/2185452571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004d800 0x7f3a1004cfd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 --2- 172.21.15.49:0/2185452571 >> v2:172.21.15.183:3300/0 conn(0x7f3a1004c170 0x7f3a1004ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 --2- 172.21.15.49:0/2185452571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d5de0 0x7f3a1004bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.746+0000 7f3a1759e640 1 -- 172.21.15.49:0/2185452571 >> 172.21.15.49:0/2185452571 conn(0x7f3a1000d190 msgr2=0x7f3a1000d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.747+0000 7f3a1759e640 1 -- 172.21.15.49:0/2185452571 shutdown_connections 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.747+0000 7f3a1759e640 1 -- 172.21.15.49:0/2185452571 wait complete. 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.749+0000 7f3a1759e640 1 Processor -- start 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.749+0000 7f3a1759e640 1 -- start start 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45654/0 (socket says 172.21.15.49:45654) 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1659c640 1 -- 172.21.15.49:0/3834552741 learned_addr learned my addr 172.21.15.49:0/3834552741 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a100d5de0 0x7f3a10199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d6440 0x7f3a101982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.753 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100dc970 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a100dc690 con 0x7f3a100d5de0 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.750+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3a100d61c0 con 0x7f3a100d6440 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 -- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d6440 msgr2=0x7f3a101982c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d6440 0x7f3a101982c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 -- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a100d5de0 msgr2=0x7f3a10199760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a100d5de0 0x7f3a10199760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10048e40 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.751+0000 7f3a1659c640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b730 tx=0x7f3a0800bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.753+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08019070 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.753+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019a9e0 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.753+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a10048880 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.753+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0800c7e0 con 0x7f3a1004c170 2024-05-09T08:58:30.754 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.753+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800cb90 con 0x7f3a1004c170 2024-05-09T08:58:30.755 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.756+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a0801c070 con 0x7f3a1004c170 2024-05-09T08:58:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.757+0000 7f3a06ffd640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 0x7f39f806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:30.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.757+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f3a0808a410 con 0x7f3a1004c170 2024-05-09T08:58:30.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.757+0000 7f3a15d9b640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 0x7f39f806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:30.758 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.759+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc005290 con 0x7f3a1004c170 2024-05-09T08:58:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.763+0000 7f3a15d9b640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 0x7f39f806e3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3a1019ade0 tx=0x7f3a0c006b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:30.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:30.769+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3a080586a0 con 0x7f3a1004c170 2024-05-09T08:58:31.019 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.020+0000 7f38ca929640 1 -- 172.21.15.49:0/408635208 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f38c40cbeb0 con 0x7f38c40b5fd0 2024-05-09T08:58:31.020 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.021+0000 7f38c0ff9640 1 -- 172.21.15.49:0/408635208 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f38b40044a0 con 0x7f38c40b5fd0 2024-05-09T08:58:31.020 INFO:teuthology.orchestra.run.smithi049.stdout:55834574881 2024-05-09T08:58:31.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.024+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 msgr2=0x7f38a406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.024+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 0x7f38a406e3f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38ac033000 tx=0x7f38ac009030 comp rx=0 tx=0).stop 2024-05-09T08:58:31.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 msgr2=0x7f38c4151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f38b400f8b0 tx=0x7f38b400fd70 comp rx=0 tx=0).stop 2024-05-09T08:58:31.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 shutdown_connections 2024-05-09T08:58:31.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38a406bf30 0x7f38a406e3f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38c414f430 0x7f38c414f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38c4154580 0x7f38c40b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 --2- 172.21.15.49:0/408635208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38c40b5fd0 0x7f38c4151070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.025+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 >> 172.21.15.49:0/408635208 conn(0x7f38c4074a20 msgr2=0x7f38c40c5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.026+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 shutdown_connections 2024-05-09T08:58:31.026 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.026+0000 7f38ba7fc640 1 -- 172.21.15.49:0/408635208 wait complete. 2024-05-09T08:58:31.036 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.037+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd92c005580 con 0x7fd94806dac0 2024-05-09T08:58:31.037 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.038+0000 7fd936ffd640 1 -- 172.21.15.49:0/3281024492 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd93c05d6c0 con 0x7fd94806dac0 2024-05-09T08:58:31.038 INFO:teuthology.orchestra.run.smithi049.stdout:279172874244 2024-05-09T08:58:31.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.041+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 msgr2=0x7fd92806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.041+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 0x7fd92806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd948154970 tx=0x7fd938009060 comp rx=0 tx=0).stop 2024-05-09T08:58:31.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.041+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 msgr2=0x7fd9480cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.041+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00e8b0 tx=0x7fd93c00ed70 comp rx=0 tx=0).stop 2024-05-09T08:58:31.041 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 shutdown_connections 2024-05-09T08:58:31.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd92806be30 0x7fd92806e2f0 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd948154970 tx=0x7fd938009060 comp rx=0 tx=0).stop 2024-05-09T08:58:31.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9480ccdb0 0x7fd94813e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9480c7910 0x7fd94813c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 --2- 172.21.15.49:0/3281024492 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd94806dac0 0x7fd9480cbc40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.042 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 >> 172.21.15.49:0/3281024492 conn(0x7fd9480cc290 msgr2=0x7fd94806c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.6 2024-05-09T08:58:31.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 shutdown_connections 2024-05-09T08:58:31.043 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.042+0000 7fd94ccd6640 1 -- 172.21.15.49:0/3281024492 wait complete. 2024-05-09T08:58:31.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f29bc005580 con 0x7f29d4006220 2024-05-09T08:58:31.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f56f804bfd0 con 0x7f56f804ba60 2024-05-09T08:58:31.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f18c00c61d0 con 0x7f18c0155190 2024-05-09T08:58:31.044 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f18b77fe640 1 -- 172.21.15.49:0/2090201843 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18ac07a2e0 con 0x7f18c0155190 2024-05-09T08:58:31.045 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:31.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f56ebfff640 1 -- 172.21.15.49:0/716365416 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56f4059250 con 0x7f56f804ba60 2024-05-09T08:58:31.045 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.044+0000 7f29c7fff640 1 -- 172.21.15.49:0/3162967846 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29dc0f7130 con 0x7f29d4006220 2024-05-09T08:58:31.046 INFO:teuthology.orchestra.run.smithi049.stdout:55834574881 2024-05-09T08:58:31.046 INFO:teuthology.orchestra.run.smithi049.stdout:55834574880 2024-05-09T08:58:31.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 msgr2=0x7f56e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 0x7f56e406e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56ec034000 tx=0x7f56ec02e000 comp rx=0 tx=0).stop 2024-05-09T08:58:31.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 msgr2=0x7f18a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.047 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 0x7f18a806e320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f18b8007fd0 tx=0x7f18b8009040 comp rx=0 tx=0).stop 2024-05-09T08:58:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 msgr2=0x7f56f817ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f56f400d800 tx=0x7f56f400dcc0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 shutdown_connections 2024-05-09T08:58:31.048 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f56e406be30 0x7f56e406e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56f80e3090 0x7f56f80cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f56f80dd3c0 0x7f56f8006ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 msgr2=0x7f18c0153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0098c0 tx=0x7f18ac0046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.049 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 --2- 172.21.15.49:0/716365416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56f804ba60 0x7f56f817ca30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 >> 172.21.15.49:0/716365416 conn(0x7f56f8050180 msgr2=0x7f56f804a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 shutdown_connections 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f18a806be60 0x7f18a806e320 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f18c015c9f0 0x7f18c0154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18c0155190 0x7f18c0153b30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.050 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 --2- 172.21.15.49:0/2090201843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f18c00cb890 0x7f18c01441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 shutdown_connections 2024-05-09T08:58:31.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 >> 172.21.15.49:0/2090201843 conn(0x7f18c0074a20 msgr2=0x7f18c015a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.051 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f5705cb3640 1 -- 172.21.15.49:0/716365416 wait complete. 2024-05-09T08:58:31.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 shutdown_connections 2024-05-09T08:58:31.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.048+0000 7f18c51b7640 1 -- 172.21.15.49:0/2090201843 wait complete. 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 msgr2=0x7f29c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 0x7f29c006e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f29d4035730 tx=0x7f29c8006b30 comp rx=0 tx=0).stop 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 msgr2=0x7f29d40dabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29dc050860 tx=0x7f29dc0692e0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 shutdown_connections 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f29c006be30 0x7f29c006e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29d40e0d50 0x7f29d4049300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29d40d7010 0x7f29d4048dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 --2- 172.21.15.49:0/3162967846 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29d4006220 0x7f29d40dabe0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.049+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 >> 172.21.15.49:0/3162967846 conn(0x7f29d404d4b0 msgr2=0x7f29d40d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.050+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 shutdown_connections 2024-05-09T08:58:31.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.050+0000 7f29e0dff640 1 -- 172.21.15.49:0/3162967846 wait complete. 2024-05-09T08:58:31.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.058+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f35d8005580 con 0x7f35f00c9c40 2024-05-09T08:58:31.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 279172874245 got 279172874244 for osd.2 2024-05-09T08:58:31.059 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.059+0000 7f35e67fc640 1 -- 172.21.15.49:0/166040599 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35ec05d340 con 0x7f35f00c9c40 2024-05-09T08:58:31.059 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:31.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.061+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7a880737e0 con 0x7f7a880c7420 2024-05-09T08:58:31.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.062+0000 7f7a6ffff640 1 -- 172.21.15.49:0/3627348228 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a8007e410 con 0x7f7a880c7420 2024-05-09T08:58:31.061 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.062+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 msgr2=0x7f35d406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 0x7f35d406e3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f35e0034000 tx=0x7f35e002e000 comp rx=0 tx=0).stop 2024-05-09T08:58:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 msgr2=0x7f35f01eabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00ebe0 tx=0x7f35ec00c520 comp rx=0 tx=0).stop 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 shutdown_connections 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f35d406bf30 0x7f35d406e3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35f015fa80 0x7f35f00c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35f0158f40 0x7f35f01f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 --2- 172.21.15.49:0/166040599 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f35f00c9c40 0x7f35f01eabc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 >> 172.21.15.49:0/166040599 conn(0x7f35f006b200 msgr2=0x7f35f0160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 shutdown_connections 2024-05-09T08:58:31.063 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.063+0000 7f35f6c2a640 1 -- 172.21.15.49:0/166040599 wait complete. 2024-05-09T08:58:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.065+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 msgr2=0x7f7a7406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.065+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 0x7f7a7406e340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a88142e90 tx=0x7f7a78008040 comp rx=0 tx=0).stop 2024-05-09T08:58:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 msgr2=0x7f7a88151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002c0d0 tx=0x7f7a80004ab0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 shutdown_connections 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7a7406be80 0x7f7a7406e340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a88159430 0x7f7a881425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a880c7c50 0x7f7a88154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 --2- 172.21.15.49:0/3627348228 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7a880c7420 0x7f7a88151610 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 >> 172.21.15.49:0/3627348228 conn(0x7f7a88074840 msgr2=0x7f7a8814f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 shutdown_connections 2024-05-09T08:58:31.067 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.066+0000 7f7a6dffb640 1 -- 172.21.15.49:0/3627348228 wait complete. 2024-05-09T08:58:31.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574881 for osd.5 2024-05-09T08:58:31.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.7 2024-05-09T08:58:31.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574880 for osd.4 2024-05-09T08:58:31.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.0 2024-05-09T08:58:31.078 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.3 2024-05-09T08:58:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.118+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f39fc005580 con 0x7f3a1004c170 2024-05-09T08:58:31.118 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.119+0000 7f3a06ffd640 1 -- 172.21.15.49:0/3834552741 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a0805c9a0 con 0x7f3a1004c170 2024-05-09T08:58:31.118 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:31.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 msgr2=0x7f39f806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.120 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 0x7f39f806e3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3a1019ade0 tx=0x7f3a0c006b30 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 msgr2=0x7f3a10199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800b730 tx=0x7f3a0800bbf0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 shutdown_connections 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f39f806bf30 0x7f39f806e3f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a100d6440 0x7f3a101982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a100d5de0 0x7f3a10199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 --2- 172.21.15.49:0/3834552741 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3a1004c170 0x7f3a10199220 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 >> 172.21.15.49:0/3834552741 conn(0x7f3a1000d190 msgr2=0x7f3a100470a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 shutdown_connections 2024-05-09T08:58:31.121 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:31.121+0000 7f3a1759e640 1 -- 172.21.15.49:0/3834552741 wait complete. 2024-05-09T08:58:31.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.1 2024-05-09T08:58:32.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T08:58:32.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T08:58:32.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T08:58:32.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T08:58:32.070 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T08:58:32.077 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T08:58:32.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T08:58:32.132 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T08:58:32.215 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.214+0000 7f4f520da640 1 Processor -- start 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.214+0000 7f4f520da640 1 -- start start 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.215+0000 7f4f520da640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4f4c159490 0x7f4f4c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.215+0000 7f4f520da640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c0c6d90 0x7f4f4c163040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f520da640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4c06b550 con 0x7f4f4c06dac0 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f520da640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4c163580 con 0x7f4f4c159490 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f520da640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c159870 con 0x7f4f4c0c6d90 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f4bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4f4c06dac0 0x7f4f4c070420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47646/0 (socket says 172.21.15.49:47646) 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f4bfff640 1 -- 172.21.15.49:0/1019774254 learned_addr learned my addr 172.21.15.49:0/1019774254 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:32.216 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.216+0000 7f4f4b7fe640 1 --2- 172.21.15.49:0/1019774254 >> v2:172.21.15.49:3300/0 conn(0x7f4f4c159490 0x7f4f4c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.217+0000 7f4f4b7fe640 1 -- 172.21.15.49:0/1019774254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c0c6d90 msgr2=0x7f4f4c163040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:58:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.217+0000 7f4f4b7fe640 1 --2- 172.21.15.49:0/1019774254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c0c6d90 0x7f4f4c163040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.217+0000 7f4f4b7fe640 1 -- 172.21.15.49:0/1019774254 >> v1:172.21.15.49:6789/0 conn(0x7f4f4c06dac0 legacy=0x7f4f4c070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.217+0000 7f4f4b7fe640 1 -- 172.21.15.49:0/1019774254 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c06b830 con 0x7f4f4c159490 2024-05-09T08:58:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.219+0000 7f4f4b7fe640 1 --2- 172.21.15.49:0/1019774254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c159490 0x7f4f4c06c730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f400097a0 tx=0x7f4f4002a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8751fceb9221f4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.220+0000 7f4f4a7fc640 1 -- 172.21.15.49:0/1019774254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40037070 con 0x7f4f4c159490 2024-05-09T08:58:32.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.220+0000 7f4f4a7fc640 1 -- 172.21.15.49:0/1019774254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f4002aca0 con 0x7f4f4c159490 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.221+0000 7f4f520da640 1 -- 172.21.15.49:0/1019774254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c159490 msgr2=0x7f4f4c06c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.221+0000 7f4f520da640 1 --2- 172.21.15.49:0/1019774254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c159490 0x7f4f4c06c730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f400097a0 tx=0x7f4f4002a010 comp rx=0 tx=0).stop 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 -- 172.21.15.49:0/1019774254 shutdown_connections 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 --2- 172.21.15.49:0/1019774254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c0c6d90 0x7f4f4c163040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 --2- 172.21.15.49:0/1019774254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c159490 0x7f4f4c06c730 secure :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f400097a0 tx=0x7f4f4002a010 comp rx=0 tx=0).stop 2024-05-09T08:58:32.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 -- 172.21.15.49:0/1019774254 >> 172.21.15.49:0/1019774254 conn(0x7f4f4c0cc290 msgr2=0x7f4f4c0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 -- 172.21.15.49:0/1019774254 shutdown_connections 2024-05-09T08:58:32.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.222+0000 7f4f520da640 1 -- 172.21.15.49:0/1019774254 wait complete. 2024-05-09T08:58:32.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.224+0000 7f4f520da640 1 Processor -- start 2024-05-09T08:58:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.225+0000 7f4f520da640 1 -- start start 2024-05-09T08:58:32.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.226+0000 7f4f520da640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 0x7f4f4c144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.226+0000 7f4f520da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 0x7f4f4c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.226+0000 7f4f520da640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.227+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 0x7f4f4c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.227+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 0x7f4f4c0c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49224/0 (socket says 172.21.15.49:49224) 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.227+0000 7f4f4b7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 0x7f4f4c144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.227+0000 7f4f4b7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 0x7f4f4c144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47730/0 (socket says 172.21.15.49:47730) 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f4bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45668/0 (socket says 172.21.15.49:45668) 2024-05-09T08:58:32.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f4affd640 1 -- 172.21.15.49:0/533991159 learned_addr learned my addr 172.21.15.49:0/533991159 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06c0d0 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06bdf0 con 0x7f4f4c0c7e30 2024-05-09T08:58:32.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.228+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c070b30 con 0x7f4f4c06dac0 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 -- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 msgr2=0x7f4f4c144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 0x7f4f4c144dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 -- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 msgr2=0x7f4f4c0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 0x7f4f4c0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c15abf0 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.229+0000 7f4f4bfff640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400cb20 tx=0x7f4f44002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.230+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44018070 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.230+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f44004d60 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.230+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400a9a0 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.231 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.231+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c15a830 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.233 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.231+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c149fd0 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.234 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.235+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f4401b070 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.235 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.236+0000 7f4f33fff640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 0x7f4f3806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.237+0000 7f4f4b7fe640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 0x7f4f3806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.237+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f4f4408a580 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.236 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.237+0000 7f4f4b7fe640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 0x7f4f3806e3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f40034000 tx=0x7f4f4002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.238+0000 7f4f31ffb640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c04f660 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.246 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.247+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4f4c15a830 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f324e812640 1 Processor -- start 2024-05-09T08:58:32.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f324e812640 1 -- start start 2024-05-09T08:58:32.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.271+0000 7f5d193a2640 1 Processor -- start 2024-05-09T08:58:32.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.271+0000 7f5d193a2640 1 -- start start 2024-05-09T08:58:32.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d193a2640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5d140c6030 0x7f5d14070270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d193a2640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d1406c0c0 0x7f5d1406e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d193a2640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d1406c4d0 con 0x7f5d140c6c90 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d193a2640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5d140c6410 con 0x7f5d140c6030 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d193a2640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f5d14168490 con 0x7f5d1406c0c0 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.272+0000 7f5d127fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5d140c6c90 0x7f5d1406dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47658/0 (socket says 172.21.15.49:47658) 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f5d127fc640 1 -- 172.21.15.49:0/3295673426 learned_addr learned my addr 172.21.15.49:0/3295673426 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f5d12ffd640 1 --2- 172.21.15.49:0/3295673426 >> v2:172.21.15.49:3300/0 conn(0x7f5d140c6030 0x7f5d14070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.273 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f5d137fe640 1 --2- 172.21.15.49:0/3295673426 >> v2:172.21.15.183:3300/0 conn(0x7f5d1406c0c0 0x7f5d1406e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f324e812640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32481515b0 0x7f32480c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7fae915f6640 1 Processor -- start 2024-05-09T08:58:32.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7fae915f6640 1 -- start start 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f324e812640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f32481575a0 0x7f32480c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f324e812640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32480c27b0 con 0x7f3248157150 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f324e812640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3248158130 con 0x7f32481515b0 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f324e812640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f32480c2dc0 con 0x7f32481575a0 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f3247fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32481515b0 0x7f32480c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f3247fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f32481515b0 0x7f32480c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45692/0 (socket says 172.21.15.49:45692) 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f3247fff640 1 -- 172.21.15.49:0/3157600383 learned_addr learned my addr 172.21.15.49:0/3157600383 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.273+0000 7f5d11ffb640 1 -- 172.21.15.49:0/3295673426 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1792288176 0 0) 0x7f5d1406c4d0 con 0x7f5d140c6c90 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f5d11ffb640 1 -- 172.21.15.49:0/3295673426 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d00005730 con 0x7f5d140c6c90 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f5d12ffd640 1 -- 172.21.15.49:0/3295673426 >> v2:172.21.15.183:3300/0 conn(0x7f5d1406c0c0 msgr2=0x7f5d1406e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f5d12ffd640 1 --2- 172.21.15.49:0/3295673426 >> v2:172.21.15.183:3300/0 conn(0x7f5d1406c0c0 0x7f5d1406e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f5d12ffd640 1 -- 172.21.15.49:0/3295673426 >> v1:172.21.15.49:6789/0 conn(0x7f5d140c6c90 legacy=0x7f5d1406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.274+0000 7f5d12ffd640 1 -- 172.21.15.49:0/3295673426 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d141686e0 con 0x7f5d140c6030 2024-05-09T08:58:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f324cdaf640 1 --2- 172.21.15.49:0/3157600383 >> v2:172.21.15.183:3300/0 conn(0x7f32481575a0 0x7f32480c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f93e1ad4640 1 Processor -- start 2024-05-09T08:58:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f93e1ad4640 1 -- start start 2024-05-09T08:58:32.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f384f4ee640 1 Processor -- start 2024-05-09T08:58:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7fae915f6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fae84050770 0x7fae840071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7fae915f6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae840076f0 0x7fae8404cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7fae915f6640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8400edf0 con 0x7fae840dd670 2024-05-09T08:58:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7fae915f6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fae840dda50 con 0x7fae840076f0 2024-05-09T08:58:32.278 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7fae915f6640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fae8404f1b0 con 0x7fae84050770 2024-05-09T08:58:32.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f3247fff640 1 -- 172.21.15.49:0/3157600383 >> v2:172.21.15.183:3300/0 conn(0x7f32481575a0 msgr2=0x7f32480c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f3247fff640 1 --2- 172.21.15.49:0/3157600383 >> v2:172.21.15.183:3300/0 conn(0x7f32481575a0 0x7f32480c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f3247fff640 1 -- 172.21.15.49:0/3157600383 >> v1:172.21.15.49:6789/0 conn(0x7f3248157150 legacy=0x7f32480c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.275+0000 7f3247fff640 1 -- 172.21.15.49:0/3157600383 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3248157b70 con 0x7f32481515b0 2024-05-09T08:58:32.279 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae840076f0 0x7fae8404cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fae840076f0 0x7fae8404cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45708/0 (socket says 172.21.15.49:45708) 2024-05-09T08:58:32.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 -- 172.21.15.49:0/1872604263 learned_addr learned my addr 172.21.15.49:0/1872604263 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f384f4ee640 1 -- start start 2024-05-09T08:58:32.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f3247fff640 1 --2- 172.21.15.49:0/3157600383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481515b0 0x7f32480c8a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32480caa00 tx=0x7f324002c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e157fa2b287aea1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f93e1ad4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc159490 0x7f93dc06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f93e1ad4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93dc0c6d90 0x7f93dc163040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f93e1ad4640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93dc06b550 con 0x7f93dc06dac0 2024-05-09T08:58:32.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93e1ad4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f93dc163580 con 0x7f93dc0c6d90 2024-05-09T08:58:32.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93e1ad4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93dc159870 con 0x7f93dc159490 2024-05-09T08:58:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 -- 172.21.15.49:0/1872604263 >> v2:172.21.15.183:3300/0 conn(0x7fae84050770 msgr2=0x7fae840071b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T08:58:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 --2- 172.21.15.49:0/1872604263 >> v2:172.21.15.183:3300/0 conn(0x7fae84050770 0x7fae840071b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 -- 172.21.15.49:0/1872604263 >> v1:172.21.15.183:6789/0 conn(0x7fae840dd670 legacy=0x7fae840e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7fae90df5640 1 -- 172.21.15.49:0/1872604263 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8404f490 con 0x7fae840076f0 2024-05-09T08:58:32.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93dbfff640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f93dc06dac0 0x7f93dc070420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49148/0 (socket says 172.21.15.49:49148) 2024-05-09T08:58:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93dbfff640 1 -- 172.21.15.49:0/2544017371 learned_addr learned my addr 172.21.15.49:0/2544017371 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93daffd640 1 --2- 172.21.15.49:0/2544017371 >> v2:172.21.15.183:3300/0 conn(0x7f93dc0c6d90 0x7f93dc163040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93db7fe640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc159490 0x7f93dc06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f38400dd3c0 0x7f3840046460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 0x7f38400e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3840045450 con 0x7f38400dd3c0 2024-05-09T08:58:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f3246ffd640 1 -- 172.21.15.49:0/3157600383 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400337d0 con 0x7f32481515b0 2024-05-09T08:58:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f3246ffd640 1 -- 172.21.15.49:0/3157600383 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240039070 con 0x7f32481515b0 2024-05-09T08:58:32.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f3246ffd640 1 -- 172.21.15.49:0/3157600383 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240033bd0 con 0x7f32481515b0 2024-05-09T08:58:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93daffd640 1 -- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc159490 msgr2=0x7f93dc06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93daffd640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc159490 0x7f93dc06c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f93daffd640 1 -- 172.21.15.49:0/2544017371 >> v1:172.21.15.183:6789/0 conn(0x7f93dc06dac0 legacy=0x7f93dc070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f93daffd640 1 -- 172.21.15.49:0/2544017371 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc06b830 con 0x7f93dc0c6d90 2024-05-09T08:58:32.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7fae90df5640 1 --2- 172.21.15.49:0/1872604263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840076f0 0x7fae8404cac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae84047c40 tx=0x7fae7802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d38351015721e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f0226c38640 1 Processor -- start 2024-05-09T08:58:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3840045fd0 con 0x7f38400e3090 2024-05-09T08:58:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384f4ee640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38400dd7a0 con 0x7f384004ba60 2024-05-09T08:58:32.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384ca89640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384ca89640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47756/0 (socket says 172.21.15.49:47756) 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384ca89640 1 -- 172.21.15.49:0/540813458 learned_addr learned my addr 172.21.15.49:0/540813458 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.277+0000 7f384d28a640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.49:3300/0 conn(0x7f38400dd3c0 0x7f3840046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f384ca89640 1 -- 172.21.15.49:0/540813458 >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 msgr2=0x7f38400e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f384da8b640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 0x7f38400e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.287 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.276+0000 7f5d12ffd640 1 --2- 172.21.15.49:0/3295673426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d140c6030 0x7f5d14070270 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d1408d520 tx=0x7f5d0c032a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b6b40a312cbb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f5d11ffb640 1 -- 172.21.15.49:0/3295673426 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c03a070 con 0x7f5d140c6030 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f5d11ffb640 1 -- 172.21.15.49:0/3295673426 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0c00ae50 con 0x7f5d140c6030 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f5d11ffb640 1 -- 172.21.15.49:0/3295673426 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c036a40 con 0x7f5d140c6030 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.278+0000 7f0226c38640 1 -- start start 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 -- 172.21.15.49:0/3157600383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481515b0 msgr2=0x7f32480c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.288 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 --2- 172.21.15.49:0/3157600383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481515b0 0x7f32480c8a10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32480caa00 tx=0x7f324002c9a0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f93daffd640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc0c6d90 0x7f93dc163040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93d00097a0 tx=0x7f93d002a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1f29483326d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f384ca89640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 0x7f38400e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f384ca89640 1 -- 172.21.15.49:0/540813458 >> v2:172.21.15.49:3300/0 conn(0x7f38400dd3c0 msgr2=0x7f3840046460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f384ca89640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.49:3300/0 conn(0x7f38400dd3c0 0x7f3840046460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f384ca89640 1 -- 172.21.15.49:0/540813458 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38400e3e90 con 0x7f384004ba60 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7fae8affd640 1 -- 172.21.15.49:0/1872604263 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78037070 con 0x7fae840076f0 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7fae8affd640 1 -- 172.21.15.49:0/1872604263 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae78032440 con 0x7fae840076f0 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7fae8affd640 1 -- 172.21.15.49:0/1872604263 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78032810 con 0x7fae840076f0 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f93da7fc640 1 -- 172.21.15.49:0/2544017371 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0037070 con 0x7f93dc0c6d90 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f93da7fc640 1 -- 172.21.15.49:0/2544017371 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d002adb0 con 0x7f93dc0c6d90 2024-05-09T08:58:32.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f93da7fc640 1 -- 172.21.15.49:0/2544017371 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0032660 con 0x7f93dc0c6d90 2024-05-09T08:58:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 -- 172.21.15.49:0/3295673426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d140c6030 msgr2=0x7f5d14070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 --2- 172.21.15.49:0/3295673426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d140c6030 0x7f5d14070270 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d1408d520 tx=0x7f5d0c032a60 comp rx=0 tx=0).stop 2024-05-09T08:58:32.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f384da8b640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 0x7f38400e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 -- 172.21.15.49:0/3295673426 shutdown_connections 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 --2- 172.21.15.49:0/3295673426 >> v2:172.21.15.183:3300/0 conn(0x7f5d1406c0c0 0x7f5d1406e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 --2- 172.21.15.49:0/3295673426 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d140c6030 0x7f5d14070270 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 -- 172.21.15.49:0/3295673426 >> 172.21.15.49:0/3295673426 conn(0x7f5d140cc290 msgr2=0x7f5d140cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 -- 172.21.15.49:0/3295673426 shutdown_connections 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f5d193a2640 1 -- 172.21.15.49:0/3295673426 wait complete. 2024-05-09T08:58:32.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 -- 172.21.15.49:0/3157600383 shutdown_connections 2024-05-09T08:58:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 --2- 172.21.15.49:0/3157600383 >> v2:172.21.15.183:3300/0 conn(0x7f32481575a0 0x7f32480c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 --2- 172.21.15.49:0/3157600383 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481515b0 0x7f32480c8a10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.279+0000 7f324e812640 1 -- 172.21.15.49:0/3157600383 >> 172.21.15.49:0/3157600383 conn(0x7f32480c9650 msgr2=0x7f32480c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f0226c38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f02200cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.293 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f324e812640 1 -- 172.21.15.49:0/3157600383 shutdown_connections 2024-05-09T08:58:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f0226c38640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f02200cb890 0x7f022015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f0226c38640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022015e410 con 0x7f0220155190 2024-05-09T08:58:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f324e812640 1 -- 172.21.15.49:0/3157600383 wait complete. 2024-05-09T08:58:32.294 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f0226c38640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f022015ae60 con 0x7f02200cb890 2024-05-09T08:58:32.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 -- 172.21.15.49:0/1872604263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840076f0 msgr2=0x7fae8404cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 --2- 172.21.15.49:0/1872604263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840076f0 0x7fae8404cac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae84047c40 tx=0x7fae7802a2c0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f0226c38640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f022015b120 con 0x7f02200b3530 2024-05-09T08:58:32.295 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f02249d4640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f0220155190 0x7f02200cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:49164/0 (socket says 172.21.15.49:49164) 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 -- 172.21.15.49:0/1872604263 shutdown_connections 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 --2- 172.21.15.49:0/1872604263 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840076f0 0x7fae8404cac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 --2- 172.21.15.49:0/1872604263 >> v2:172.21.15.183:3300/0 conn(0x7fae84050770 0x7fae840071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 -- 172.21.15.49:0/1872604263 >> 172.21.15.49:0/1872604263 conn(0x7fae8400e160 msgr2=0x7fae8400e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f02249d4640 1 -- 172.21.15.49:0/756439189 learned_addr learned my addr 172.21.15.49:0/756439189 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:32.296 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f021ffff640 1 --2- 172.21.15.49:0/756439189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f02200cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 -- 172.21.15.49:0/1872604263 shutdown_connections 2024-05-09T08:58:32.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7f02251d5640 1 --2- 172.21.15.49:0/756439189 >> v2:172.21.15.183:3300/0 conn(0x7f02200cb890 0x7f022015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.297 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f384ca89640 1 --2- 172.21.15.49:0/540813458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f384802d750 tx=0x7f38480a6bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5168a468946259cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f383ffff640 1 -- 172.21.15.49:0/540813458 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480a89e0 con 0x7f384004ba60 2024-05-09T08:58:32.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f383ffff640 1 -- 172.21.15.49:0/540813458 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38480a8b80 con 0x7f384004ba60 2024-05-09T08:58:32.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f324e812640 1 Processor -- start 2024-05-09T08:58:32.298 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f383ffff640 1 -- 172.21.15.49:0/540813458 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480a7440 con 0x7f384004ba60 2024-05-09T08:58:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f324e812640 1 -- start start 2024-05-09T08:58:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f5d193a2640 1 Processor -- start 2024-05-09T08:58:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f5d193a2640 1 -- start start 2024-05-09T08:58:32.299 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 -- 172.21.15.49:0/540813458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 msgr2=0x7f384004be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 --2- 172.21.15.49:0/540813458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f384802d750 tx=0x7f38480a6bc0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 -- 172.21.15.49:0/540813458 shutdown_connections 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.183:3300/0 conn(0x7f38400e3090 0x7f38400e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 --2- 172.21.15.49:0/540813458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f384004ba60 0x7f384004be80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 --2- 172.21.15.49:0/540813458 >> v2:172.21.15.49:3300/0 conn(0x7f38400dd3c0 0x7f3840046460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 -- 172.21.15.49:0/540813458 >> 172.21.15.49:0/540813458 conn(0x7f3840050180 msgr2=0x7f3840050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.300 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.280+0000 7fae915f6640 1 -- 172.21.15.49:0/1872604263 wait complete. 2024-05-09T08:58:32.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7fae915f6640 1 Processor -- start 2024-05-09T08:58:32.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 -- 172.21.15.49:0/540813458 shutdown_connections 2024-05-09T08:58:32.301 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f384f4ee640 1 -- 172.21.15.49:0/540813458 wait complete. 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f021ffff640 1 -- 172.21.15.49:0/756439189 >> v2:172.21.15.183:3300/0 conn(0x7f02200cb890 msgr2=0x7f022015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f021ffff640 1 --2- 172.21.15.49:0/756439189 >> v2:172.21.15.183:3300/0 conn(0x7f02200cb890 0x7f022015e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f021ffff640 1 -- 172.21.15.49:0/756439189 >> v1:172.21.15.183:6789/0 conn(0x7f0220155190 legacy=0x7f02200cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.282+0000 7f021ffff640 1 -- 172.21.15.49:0/756439189 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022015b400 con 0x7f02200b3530 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f021ffff640 1 --2- 172.21.15.49:0/756439189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f02200cb350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02100097a0 tx=0x7f021002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bdb52a808b29d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f93e1ad4640 1 -- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc0c6d90 msgr2=0x7f93dc163040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.302 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.281+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc0c6d90 0x7f93dc163040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93d00097a0 tx=0x7f93d002a0a0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7fae915f6640 1 -- start start 2024-05-09T08:58:32.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32481515b0 0x7f324814af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f93e1ad4640 1 -- 172.21.15.49:0/2544017371 shutdown_connections 2024-05-09T08:58:32.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc0c6d90 0x7f93dc163040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.303 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/2544017371 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc159490 0x7f93dc06c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f93e1ad4640 1 -- 172.21.15.49:0/2544017371 >> 172.21.15.49:0/2544017371 conn(0x7f93dc0cc290 msgr2=0x7f93dc0cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 0x7f3248072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f32480c2a50 con 0x7f32481575a0 2024-05-09T08:58:32.304 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32480c8f50 con 0x7f3248157150 2024-05-09T08:58:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f5d193a2640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 0x7f5d14161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f324e812640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f32480c81b0 con 0x7f32481515b0 2024-05-09T08:58:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f32477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f32477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49306/0 (socket says 172.21.15.49:49306) 2024-05-09T08:58:32.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.283+0000 7f32477fe640 1 -- 172.21.15.49:0/3158171208 learned_addr learned my addr 172.21.15.49:0/3158171208 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f93e1ad4640 1 -- 172.21.15.49:0/2544017371 shutdown_connections 2024-05-09T08:58:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f93e1ad4640 1 -- 172.21.15.49:0/2544017371 wait complete. 2024-05-09T08:58:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f021f7fe640 1 -- 172.21.15.49:0/756439189 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210037070 con 0x7f02200b3530 2024-05-09T08:58:32.306 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f021f7fe640 1 -- 172.21.15.49:0/756439189 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021002ada0 con 0x7f02200b3530 2024-05-09T08:58:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f021f7fe640 1 -- 172.21.15.49:0/756439189 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210032660 con 0x7f02200b3530 2024-05-09T08:58:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f384f4ee640 1 Processor -- start 2024-05-09T08:58:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae84050770 0x7fae84173f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.307 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840dd670 0x7fae84185750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f384f4ee640 1 -- start start 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fae8404fd30 con 0x7fae840dd670 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae8404fa50 con 0x7fae84050770 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae915f6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fae84050010 con 0x7fae840076f0 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae8bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae8bfff640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47776/0 (socket says 172.21.15.49:47776) 2024-05-09T08:58:32.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae8bfff640 1 -- 172.21.15.49:0/1574775098 learned_addr learned my addr 172.21.15.49:0/1574775098 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f324cdaf640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 0x7f3248072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 -- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32481515b0 msgr2=0x7f324814af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32481515b0 0x7f324814af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 -- 172.21.15.49:0/3158171208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 msgr2=0x7f3248072670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 0x7f3248072670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 -- 172.21.15.49:0/3158171208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324806c5a0 con 0x7f3248157150 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f324cdaf640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 0x7f3248072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7f32477fe640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c460 tx=0x7f323c00c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c016020 con 0x7f3248157150 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 -- 172.21.15.49:0/756439189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 msgr2=0x7f02200cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 --2- 172.21.15.49:0/756439189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f02200cb350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02100097a0 tx=0x7f021002a110 comp rx=0 tx=0).stop 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d193a2640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 0x7f5d1415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d12ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 0x7f5d14161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d193a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d140c6c90 0x7f5d1415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d193a2640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1406b8c0 con 0x7f5d1406c0c0 2024-05-09T08:58:32.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d193a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d141689c0 con 0x7f5d140c6c90 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d193a2640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5d1406bba0 con 0x7f5d140c6030 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d12ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 0x7f5d14161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45742/0 (socket says 172.21.15.49:45742) 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d12ffd640 1 -- 172.21.15.49:0/2891126687 learned_addr learned my addr 172.21.15.49:0/2891126687 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 0x7f5d1415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 -- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d140c6c90 msgr2=0x7f5d1415e010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d140c6c90 0x7f5d1415e010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 -- 172.21.15.49:0/2891126687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 msgr2=0x7f5d14161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 0x7f5d14161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f5d127fc640 1 -- 172.21.15.49:0/2891126687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d14201730 con 0x7f5d140c6030 2024-05-09T08:58:32.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f5d127fc640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 0x7f5d1415dad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400cad0 tx=0x7f5d0400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.284+0000 7fae8b7fe640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae84050770 0x7fae84173f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae90df5640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840dd670 0x7fae84185750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 -- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae84050770 msgr2=0x7fae84173f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae84050770 0x7fae84173f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 -- 172.21.15.49:0/1574775098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840dd670 msgr2=0x7fae84185750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840dd670 0x7fae84185750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 -- 172.21.15.49:0/1574775098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae84173160 con 0x7fae840076f0 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae8bfff640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae8000b420 tx=0x7fae8000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32481ed7b0 con 0x7f3248157150 2024-05-09T08:58:32.312 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c012870 con 0x7f3248157150 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 -- 172.21.15.49:0/756439189 shutdown_connections 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 --2- 172.21.15.49:0/756439189 >> v2:172.21.15.183:3300/0 conn(0x7f02200cb890 0x7f022015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 --2- 172.21.15.49:0/756439189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f02200cb350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f0226c38640 1 -- 172.21.15.49:0/756439189 >> 172.21.15.49:0/756439189 conn(0x7f0220074a20 msgr2=0x7f0220074e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384f4ee640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.313 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384f4ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38400dd3c0 0x7f384004f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384d28a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384d28a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:45756/0 (socket says 172.21.15.49:45756) 2024-05-09T08:58:32.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.285+0000 7f93e1ad4640 1 Processor -- start 2024-05-09T08:58:32.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f93e1ad4640 1 -- start start 2024-05-09T08:58:32.314 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04017070 con 0x7f5d140c6030 2024-05-09T08:58:32.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384d28a640 1 -- 172.21.15.49:0/3486325572 learned_addr learned my addr 172.21.15.49:0/3486325572 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0400ac30 con 0x7f5d140c6030 2024-05-09T08:58:32.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80011830 con 0x7fae840076f0 2024-05-09T08:58:32.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f5d193a2640 1 -- 172.21.15.49:0/2891126687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d141f14c0 con 0x7f5d140c6030 2024-05-09T08:58:32.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 0x7f38400cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384ca89640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38400dd3c0 0x7f384004f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f5d193a2640 1 -- 172.21.15.49:0/2891126687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d14161f40 con 0x7f5d140c6030 2024-05-09T08:58:32.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3840045c60 con 0x7f384004ba60 2024-05-09T08:58:32.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3840045980 con 0x7f38400dd3c0 2024-05-09T08:58:32.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80016070 con 0x7fae840076f0 2024-05-09T08:58:32.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80011c10 con 0x7fae840076f0 2024-05-09T08:58:32.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f0226c38640 1 -- 172.21.15.49:0/756439189 shutdown_connections 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f0226c38640 1 -- 172.21.15.49:0/756439189 wait complete. 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384da8b640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 0x7f38400cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3840005e90 con 0x7f38400e3090 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384da8b640 1 -- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38400dd3c0 msgr2=0x7f384004f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384da8b640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38400dd3c0 0x7f384004f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f384da8b640 1 -- 172.21.15.49:0/3486325572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 msgr2=0x7f38400da650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f384da8b640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f384da8b640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3840178da0 con 0x7f38400e3090 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93dc0c6d90 0x7f93dc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc06c0d0 con 0x7f93dc0c6d90 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc06bdf0 con 0x7f93dc159490 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93e1ad4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f93dc070b30 con 0x7f93dc06dac0 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93db7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93db7fe640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47804/0 (socket says 172.21.15.49:47804) 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49330/0 (socket says 172.21.15.49:49330) 2024-05-09T08:58:32.319 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93dbfff640 1 -- 172.21.15.49:0/3807500830 learned_addr learned my addr 172.21.15.49:0/3807500830 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.287+0000 7f93daffd640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93dc0c6d90 0x7f93dc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04012400 con 0x7f5d140c6030 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 -- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 msgr2=0x7f93dc156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 -- 172.21.15.49:0/3807500830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93dc0c6d90 msgr2=0x7f93dc1553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93dc0c6d90 0x7f93dc1553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.320 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93db7fe640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:32.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 -- 172.21.15.49:0/3807500830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc159eb0 con 0x7f93dc159490 2024-05-09T08:58:32.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f384da8b640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 0x7f38400cc1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f384400c5c0 tx=0x7f384400ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f384d28a640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:58:32.321 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f93dbfff640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93d400cb20 tx=0x7f93d400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae84173440 con 0x7fae840076f0 2024-05-09T08:58:32.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae84173a00 con 0x7fae840076f0 2024-05-09T08:58:32.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844015070 con 0x7f38400e3090 2024-05-09T08:58:32.322 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d400aca0 con 0x7f93dc159490 2024-05-09T08:58:32.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4012400 con 0x7f93dc159490 2024-05-09T08:58:32.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc1f4eb0 con 0x7f93dc159490 2024-05-09T08:58:32.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3844011800 con 0x7f38400e3090 2024-05-09T08:58:32.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d400ae60 con 0x7f93dc159490 2024-05-09T08:58:32.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc15a450 con 0x7f93dc159490 2024-05-09T08:58:32.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d0401a070 con 0x7f5d140c6030 2024-05-09T08:58:32.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844011c10 con 0x7f38400e3090 2024-05-09T08:58:32.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38400d8fd0 con 0x7f38400e3090 2024-05-09T08:58:32.324 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f0226c38640 1 Processor -- start 2024-05-09T08:58:32.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.289+0000 7f0226c38640 1 -- start start 2024-05-09T08:58:32.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f384017d0a0 con 0x7f38400e3090 2024-05-09T08:58:32.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f0226c38640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.325 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f0226c38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.291+0000 7f0226c38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 0x7f0220146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.286+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324815aca0 con 0x7f3248157150 2024-05-09T08:58:32.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.288+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c012c60 con 0x7f3248157150 2024-05-09T08:58:32.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.291+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f323c019020 con 0x7f3248157150 2024-05-09T08:58:32.326 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.291+0000 7f3244ff9640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 0x7f323406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.291+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae8001b460 con 0x7fae840076f0 2024-05-09T08:58:32.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f3247fff640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 0x7f323406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.327 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3844018070 con 0x7f38400e3090 2024-05-09T08:58:32.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3840005390 con 0x7f38400e3090 2024-05-09T08:58:32.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f323c012e00 con 0x7f3248157150 2024-05-09T08:58:32.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.290+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec005290 con 0x7f5d140c6030 2024-05-09T08:58:32.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324804f660 con 0x7f3248157150 2024-05-09T08:58:32.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f021ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f021ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:49338/0 (socket says 172.21.15.49:49338) 2024-05-09T08:58:32.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc005290 con 0x7f93dc159490 2024-05-09T08:58:32.329 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93d40127f0 con 0x7f93dc159490 2024-05-09T08:58:32.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7fae88ff9640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 0x7fae7006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.292+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fae8008a9e0 con 0x7fae840076f0 2024-05-09T08:58:32.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8404bd20 con 0x7fae840076f0 2024-05-09T08:58:32.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f02251d5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 0x7f0220146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f383dffb640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 0x7f382c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f02249d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f02249d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47806/0 (socket says 172.21.15.49:47806) 2024-05-09T08:58:32.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f384408ace0 con 0x7f38400e3090 2024-05-09T08:58:32.331 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f384d28a640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 0x7f382c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7fae8b7fe640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 0x7fae7006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f0226c38640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f022015bca0 con 0x7f0220155190 2024-05-09T08:58:32.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f021ffff640 1 -- 172.21.15.49:0/145723216 learned_addr learned my addr 172.21.15.49:0/145723216 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.332 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f93c3fff640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 0x7f93c806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f022015b9c0 con 0x7f02200cb890 2024-05-09T08:58:32.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0220155570 con 0x7f02200b3530 2024-05-09T08:58:32.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f93d408b4e0 con 0x7f93dc159490 2024-05-09T08:58:32.333 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7f93db7fe640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 0x7f93c806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.293+0000 7fae8b7fe640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 0x7fae7006e3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae8c050ef0 tx=0x7fae8c06ae60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 -- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 msgr2=0x7f0220141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 -- 172.21.15.49:0/145723216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 msgr2=0x7f0220146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.334 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f384d28a640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 0x7f382c06e3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3838009f70 tx=0x7f3838009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f5d0b7fe640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 0x7f5cf406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f02249d4640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-09T08:58:32.335 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f93db7fe640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 0x7f93c806e3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007aa0 tx=0x7f93cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 0x7f0220146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02200b50d0 con 0x7f02200cb890 2024-05-09T08:58:32.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f5d12ffd640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 0x7f5cf406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f3247fff640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 0x7f323406e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3240038000 tx=0x7f3240033000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f02251d5640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 0x7f0220146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:32.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f5d0408ab40 con 0x7f5d140c6030 2024-05-09T08:58:32.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.294+0000 7f021ffff640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02100098c0 tx=0x7f02100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.295+0000 7f5d12ffd640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 0x7f5cf406e3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0075d0 tx=0x7f5d0c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.298+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3844058f70 con 0x7f38400e3090 2024-05-09T08:58:32.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.298+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f323c058b60 con 0x7f3248157150 2024-05-09T08:58:32.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.302+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fae8404bd20 con 0x7fae840076f0 2024-05-09T08:58:32.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.303+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f93d40597b0 con 0x7f93dc159490 2024-05-09T08:58:32.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.302+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210037050 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.303+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0210038040 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.303+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02100044b0 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.303+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02201417c0 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.303+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02200b5350 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.305+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0210003610 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.305+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0200005290 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.306+0000 7f021d7fa640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 0x7f020c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.306+0000 7f02249d4640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 0x7f020c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.306+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f02100ab470 con 0x7f02200cb890 2024-05-09T08:58:32.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.307+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5d04058e60 con 0x7f5d140c6030 2024-05-09T08:58:32.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.307+0000 7f02249d4640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 0x7f020c06e370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0214000f60 tx=0x7f0214009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.311+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02201417c0 con 0x7f02200cb890 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.312+0000 7f8e83b10640 1 Processor -- start 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.312+0000 7f8e83b10640 1 -- start start 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.313+0000 7f8e83b10640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8e7c154b90 0x7f8e7c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.313+0000 7f8e83b10640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e7c072600 con 0x7f8e7c0c6d50 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.313+0000 7f8e83b10640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e7c0728e0 con 0x7f8e7c154f70 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.313+0000 7f8e83b10640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f8e7c0c5880 con 0x7f8e7c154b90 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.314+0000 7f8e820ad640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8e7c0c6d50 0x7f8e7c071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:47678/0 (socket says 172.21.15.49:47678) 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.314+0000 7f8e820ad640 1 -- 172.21.15.49:0/4194610257 learned_addr learned my addr 172.21.15.49:0/4194610257 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.314+0000 7f8e818ac640 1 --2- 172.21.15.49:0/4194610257 >> v2:172.21.15.183:3300/0 conn(0x7f8e7c154b90 0x7f8e7c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.315+0000 7f8e818ac640 1 -- 172.21.15.49:0/4194610257 >> v1:172.21.15.183:6789/0 conn(0x7f8e7c154f70 legacy=0x7f8e7c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.315+0000 7f8e818ac640 1 -- 172.21.15.49:0/4194610257 >> v1:172.21.15.49:6789/0 conn(0x7f8e7c0c6d50 legacy=0x7f8e7c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.315+0000 7f8e818ac640 1 -- 172.21.15.49:0/4194610257 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c0c5b60 con 0x7f8e7c154b90 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.316+0000 7f8e818ac640 1 --2- 172.21.15.49:0/4194610257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cb280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0097a0 tx=0x7f8e6c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f620f47602ecf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.316+0000 7f8e808aa640 1 -- 172.21.15.49:0/4194610257 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c037070 con 0x7f8e7c154b90 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.316+0000 7f8e808aa640 1 -- 172.21.15.49:0/4194610257 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6c032440 con 0x7f8e7c154b90 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.316+0000 7f8e808aa640 1 -- 172.21.15.49:0/4194610257 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c032810 con 0x7f8e7c154b90 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.317+0000 7f8e83b10640 1 -- 172.21.15.49:0/4194610257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154b90 msgr2=0x7f8e7c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.317+0000 7f8e83b10640 1 --2- 172.21.15.49:0/4194610257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cb280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0097a0 tx=0x7f8e6c02a210 comp rx=0 tx=0).stop 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.318+0000 7f8e83b10640 1 -- 172.21.15.49:0/4194610257 shutdown_connections 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.318+0000 7f8e83b10640 1 --2- 172.21.15.49:0/4194610257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cb280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.318+0000 7f8e83b10640 1 -- 172.21.15.49:0/4194610257 >> 172.21.15.49:0/4194610257 conn(0x7f8e7c074840 msgr2=0x7f8e7c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.318+0000 7f8e83b10640 1 -- 172.21.15.49:0/4194610257 shutdown_connections 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.318+0000 7f8e83b10640 1 -- 172.21.15.49:0/4194610257 wait complete. 2024-05-09T08:58:32.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.320+0000 7f8e83b10640 1 Processor -- start 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.321+0000 7f8e83b10640 1 -- start start 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.322+0000 7f8e83b10640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 0x7f8e7c0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.322+0000 7f8e83b10640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.322+0000 7f8e83b10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 0x7f8e7c1fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e818ac640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 0x7f8e7c0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e818ac640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 0x7f8e7c0c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:47818/0 (socket says 172.21.15.49:47818) 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e818ac640 1 -- 172.21.15.49:0/1504644818 learned_addr learned my addr 172.21.15.49:0/1504644818 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 0x7f8e7c1fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c15af40 con 0x7f8e7c154b90 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e7c0c6120 con 0x7f8e7c154f70 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8e7c0cb7c0 con 0x7f8e7c0c6d50 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 -- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 msgr2=0x7f8e7c0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 0x7f8e7c0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 -- 172.21.15.49:0/1504644818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c154b90 msgr2=0x7f8e7c0cc270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cc270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.323+0000 7f8e820ad640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7c0ce430 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.324+0000 7f8e820ad640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 0x7f8e7c1fd9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400cb20 tx=0x7f8e7400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.324+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74017070 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.325+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7c0ce6d0 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.325+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e7c0cedb0 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.325+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7400abf0 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.325+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74012400 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.327+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e7401a070 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.330+0000 7f8e727fc640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 0x7f8e6006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.330+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e64005290 con 0x7f8e7c154f70 2024-05-09T08:58:32.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.330+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f8e7408ac60 con 0x7f8e7c154f70 2024-05-09T08:58:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.330+0000 7f8e818ac640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 0x7f8e6006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.331+0000 7f8e818ac640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 0x7f8e6006e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02a250 tx=0x7f8e6c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.350 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.335+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8e74058ef0 con 0x7f8e7c154f70 2024-05-09T08:58:32.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.654+0000 7f4f31ffb640 1 -- 172.21.15.49:0/533991159 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4f4c15caf0 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.655+0000 7f4f33fff640 1 -- 172.21.15.49:0/533991159 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f44058810 con 0x7f4f4c15b0a0 2024-05-09T08:58:32.655 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:32.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.659+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 msgr2=0x7f4f3806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.659+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 0x7f4f3806e3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4f40034000 tx=0x7f4f4002e000 comp rx=0 tx=0).stop 2024-05-09T08:58:32.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.659+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 msgr2=0x7f4f4c15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.659 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.659+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400cb20 tx=0x7f4f44002170 comp rx=0 tx=0).stop 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 shutdown_connections 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f4f3806bf30 0x7f4f3806e3f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4f4c15b0a0 0x7f4f4c15b550 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c0c7e30 0x7f4f4c0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 --2- 172.21.15.49:0/533991159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4f4c06dac0 0x7f4f4c144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.660+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 >> 172.21.15.49:0/533991159 conn(0x7f4f4c0cc290 msgr2=0x7f4f4c147960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.661+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 shutdown_connections 2024-05-09T08:58:32.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.661+0000 7f4f520da640 1 -- 172.21.15.49:0/533991159 wait complete. 2024-05-09T08:58:32.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.6 2024-05-09T08:58:32.677 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.703+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5cec005580 con 0x7f5d140c6030 2024-05-09T08:58:32.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.703+0000 7f5d0b7fe640 1 -- 172.21.15.49:0/2891126687 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d0405d160 con 0x7f5d140c6030 2024-05-09T08:58:32.703 INFO:teuthology.orchestra.run.smithi049.stdout:279172874245 2024-05-09T08:58:32.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 msgr2=0x7f5cf406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 0x7f5cf406e3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0075d0 tx=0x7f5d0c00a000 comp rx=0 tx=0).stop 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 msgr2=0x7f5d1415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 0x7f5d1415dad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400cad0 tx=0x7f5d0400cf90 comp rx=0 tx=0).stop 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 shutdown_connections 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5cf406bf00 0x7f5cf406e3c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d140c6c90 0x7f5d1415e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d140c6030 0x7f5d1415dad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 --2- 172.21.15.49:0/2891126687 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5d1406c0c0 0x7f5d14161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 >> 172.21.15.49:0/2891126687 conn(0x7f5d140cc290 msgr2=0x7f5d1406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 shutdown_connections 2024-05-09T08:58:32.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.706+0000 7f5d097fa640 1 -- 172.21.15.49:0/2891126687 wait complete. 2024-05-09T08:58:32.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 279172874245 got 279172874245 for osd.2 2024-05-09T08:58:32.717 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.732+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f384004b840 con 0x7f38400e3090 2024-05-09T08:58:32.731 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.732+0000 7f383dffb640 1 -- 172.21.15.49:0/3486325572 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f384405d270 con 0x7f38400e3090 2024-05-09T08:58:32.732 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:32.732 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.734+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f32480c20e0 con 0x7f3248157150 2024-05-09T08:58:32.733 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.734+0000 7f3244ff9640 1 -- 172.21.15.49:0/3158171208 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f323c058460 con 0x7f3248157150 2024-05-09T08:58:32.733 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 msgr2=0x7f382c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 0x7f382c06e3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3838009f70 tx=0x7f3838009190 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 msgr2=0x7f38400cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 0x7f38400cc1d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f384400c5c0 tx=0x7f384400ca80 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 shutdown_connections 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f382c06bf30 0x7f382c06e3f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38400e3090 0x7f38400cc1d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38400dd3c0 0x7f384004f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 --2- 172.21.15.49:0/3486325572 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f384004ba60 0x7f38400da650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 >> 172.21.15.49:0/3486325572 conn(0x7f3840050180 msgr2=0x7f38400462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 shutdown_connections 2024-05-09T08:58:32.735 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f384f4ee640 1 -- 172.21.15.49:0/3486325572 wait complete. 2024-05-09T08:58:32.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.735+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f93bc005580 con 0x7f93dc159490 2024-05-09T08:58:32.736 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.736+0000 7f93c3fff640 1 -- 172.21.15.49:0/3807500830 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f93d405dab0 con 0x7f93dc159490 2024-05-09T08:58:32.736 INFO:teuthology.orchestra.run.smithi049.stdout:55834574884 2024-05-09T08:58:32.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 msgr2=0x7f323406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fae840e3a40 con 0x7fae840076f0 2024-05-09T08:58:32.737 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 0x7f323406e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3240038000 tx=0x7f3240033000 comp rx=0 tx=0).stop 2024-05-09T08:58:32.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 msgr2=0x7f3248072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c460 tx=0x7f323c00c920 comp rx=0 tx=0).stop 2024-05-09T08:58:32.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 msgr2=0x7f93c806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 0x7f93c806e3c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007aa0 tx=0x7f93cc008040 comp rx=0 tx=0).stop 2024-05-09T08:58:32.738 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 msgr2=0x7f93dc161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93d400cb20 tx=0x7f93d400a060 comp rx=0 tx=0).stop 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 shutdown_connections 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 shutdown_connections 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f323406beb0 0x7f323406e370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f32481575a0 0x7f3248072670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.739 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f93c806bf00 0x7f93c806e3c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3248157150 0x7f3248072130 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 --2- 172.21.15.49:0/3158171208 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32481515b0 0x7f324814af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.740 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 >> 172.21.15.49:0/3158171208 conn(0x7f32480c9650 msgr2=0x7f32480cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93dc159490 0x7f93dc161300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f93dc0c6d90 0x7f93dc1553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 --2- 172.21.15.49:0/3807500830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93dc06dac0 0x7f93dc156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 >> 172.21.15.49:0/3807500830 conn(0x7f93dc0cc290 msgr2=0x7f93dc1575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 shutdown_connections 2024-05-09T08:58:32.741 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f324e812640 1 -- 172.21.15.49:0/3158171208 wait complete. 2024-05-09T08:58:32.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 shutdown_connections 2024-05-09T08:58:32.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7f93e1ad4640 1 -- 172.21.15.49:0/3807500830 wait complete. 2024-05-09T08:58:32.742 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.738+0000 7fae88ff9640 1 -- 172.21.15.49:0/1574775098 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae80020020 con 0x7fae840076f0 2024-05-09T08:58:32.743 INFO:teuthology.orchestra.run.smithi049.stdout:55834574882 2024-05-09T08:58:32.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.741+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0200005850 con 0x7f02200cb890 2024-05-09T08:58:32.743 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7f021d7fa640 1 -- 172.21.15.49:0/145723216 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02200b50d0 con 0x7f02200cb890 2024-05-09T08:58:32.743 INFO:teuthology.orchestra.run.smithi049.stdout:55834574881 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 msgr2=0x7fae7006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 0x7fae7006e3c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae8c050ef0 tx=0x7fae8c06ae60 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 msgr2=0x7fae84006de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae8000b420 tx=0x7fae8000b8e0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 shutdown_connections 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fae7006bf00 0x7fae7006e3c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fae840dd670 0x7fae84185750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae84050770 0x7fae84173f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 --2- 172.21.15.49:0/1574775098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fae840076f0 0x7fae84006de0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.742+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 >> 172.21.15.49:0/1574775098 conn(0x7fae8400e160 msgr2=0x7fae84053710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.743+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 shutdown_connections 2024-05-09T08:58:32.745 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.743+0000 7fae915f6640 1 -- 172.21.15.49:0/1574775098 wait complete. 2024-05-09T08:58:32.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.744+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 msgr2=0x7f020c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.744+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 0x7f020c06e370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0214000f60 tx=0x7f0214009040 comp rx=0 tx=0).stop 2024-05-09T08:58:32.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.744+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 msgr2=0x7f0220145c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.744+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02100098c0 tx=0x7f02100046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.746 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 shutdown_connections 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f020c06beb0 0x7f020c06e370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0220155190 0x7f0220146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02200cb890 0x7f0220145c50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 --2- 172.21.15.49:0/145723216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02200b3530 0x7f0220141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 >> 172.21.15.49:0/145723216 conn(0x7f0220074a20 msgr2=0x7f02200c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 shutdown_connections 2024-05-09T08:58:32.747 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.745+0000 7f0226c38640 1 -- 172.21.15.49:0/145723216 wait complete. 2024-05-09T08:58:32.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.0 2024-05-09T08:58:32.750 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.756 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.757+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8e64005580 con 0x7f8e7c154f70 2024-05-09T08:58:32.756 INFO:teuthology.orchestra.run.smithi049.stdout:55834574883 2024-05-09T08:58:32.757 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.757+0000 7f8e727fc640 1 -- 172.21.15.49:0/1504644818 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e7405d1f0 con 0x7f8e7c154f70 2024-05-09T08:58:32.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.7 2024-05-09T08:58:32.759 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574881 for osd.4 2024-05-09T08:58:32.760 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.3 2024-05-09T08:58:32.761 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 msgr2=0x7f8e6006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 0x7f8e6006e3c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c02a250 tx=0x7f8e6c02e000 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 msgr2=0x7f8e7c1fd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 0x7f8e7c1fd9a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400cb20 tx=0x7f8e7400cf20 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 shutdown_connections 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8e6006bf00 0x7f8e6006e3c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e7c154f70 0x7f8e7c1fd9a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8e7c154b90 0x7f8e7c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 --2- 172.21.15.49:0/1504644818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8e7c0c6d50 0x7f8e7c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 >> 172.21.15.49:0/1504644818 conn(0x7f8e7c074840 msgr2=0x7f8e7c074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 shutdown_connections 2024-05-09T08:58:32.762 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.760+0000 7f8e83b10640 1 -- 172.21.15.49:0/1504644818 wait complete. 2024-05-09T08:58:32.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574882 got 55834574882 for osd.5 2024-05-09T08:58:32.763 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574883 for osd.1 2024-05-09T08:58:32.770 DEBUG:teuthology.parallel:result is None 2024-05-09T08:58:32.771 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-09T08:58:32.771 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:32.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 Processor -- start 2024-05-09T08:58:32.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 -- start start 2024-05-09T08:58:32.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2bf0072300 0x7f2bf00726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf00c66c0 0x7f2bf01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.916 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2bf00c6e00 0x7f2bf006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf00c3900 con 0x7f2bf00c6e00 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf0158f20 con 0x7f2bf0072300 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.915+0000 7f2bf62d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf01598f0 con 0x7f2bf00c66c0 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2bf00c6e00 0x7f2bf006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2bf00c6e00 0x7f2bf006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52660/0 (socket says 172.21.15.49:52660) 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 -- 172.21.15.49:0/3250124717 learned_addr learned my addr 172.21.15.49:0/3250124717 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bef7fe640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf00c66c0 0x7f2bf01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2beffff640 1 --2- 172.21.15.49:0/3250124717 >> v2:172.21.15.183:3300/0 conn(0x7f2bf0072300 0x7f2bf00726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 -- 172.21.15.49:0/3250124717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf00c66c0 msgr2=0x7f2bf01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf00c66c0 0x7f2bf01593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 -- 172.21.15.49:0/3250124717 >> v2:172.21.15.183:3300/0 conn(0x7f2bf0072300 msgr2=0x7f2bf00726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3250124717 >> v2:172.21.15.183:3300/0 conn(0x7f2bf0072300 0x7f2bf00726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.917 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.916+0000 7f2bf4871640 1 -- 172.21.15.49:0/3250124717 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf006e980 con 0x7f2bf00c6e00 2024-05-09T08:58:32.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf006e440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00754c0 tx=0x7f2be802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3b833cfd5e8205 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2beeffd640 1 -- 172.21.15.49:0/3250124717 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8036070 con 0x7f2bf00c6e00 2024-05-09T08:58:32.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2beeffd640 1 -- 172.21.15.49:0/3250124717 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be8031600 con 0x7f2bf00c6e00 2024-05-09T08:58:32.918 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2beeffd640 1 -- 172.21.15.49:0/3250124717 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be80319d0 con 0x7f2bf00c6e00 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3250124717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 msgr2=0x7f2bf006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.917+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf006e440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00754c0 tx=0x7f2be802a6a0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3250124717 shutdown_connections 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf006e440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3250124717 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf00c66c0 0x7f2bf01593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3250124717 >> v2:172.21.15.183:3300/0 conn(0x7f2bf0072300 0x7f2bf00726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3250124717 >> 172.21.15.49:0/3250124717 conn(0x7f2bf0074840 msgr2=0x7f2bf0074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:32.921 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3250124717 shutdown_connections 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.918+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3250124717 wait complete. 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.919+0000 7f2bf62d4640 1 Processor -- start 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.919+0000 7f2bf62d4640 1 -- start start 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf0072300 0x7f2bf00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf00c66c0 0x7f2bf00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00c3e30 con 0x7f2bf00c6e00 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf00c3b50 con 0x7f2bf00c66c0 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf62d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf0158bb0 con 0x7f2bf0072300 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52662/0 (socket says 172.21.15.49:52662) 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 -- 172.21.15.49:0/3497706170 learned_addr learned my addr 172.21.15.49:0/3497706170 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:32.922 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bef7fe640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf00c66c0 0x7f2bf00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2beffff640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf0072300 0x7f2bf00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 -- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf0072300 msgr2=0x7f2bf00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf0072300 0x7f2bf00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 -- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf00c66c0 msgr2=0x7f2bf00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf00c66c0 0x7f2bf00cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 -- 172.21.15.49:0/3497706170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf00708f0 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2bf4871640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034040 tx=0x7f2be8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8041020 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.920+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be80077a0 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.921+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf00ccbc0 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.921+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8036040 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.921+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf00cd180 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.921+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0068560 con 0x7f2bf00c6e00 2024-05-09T08:58:32.923 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.922+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2be8044020 con 0x7f2bf00c6e00 2024-05-09T08:58:32.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.923+0000 7f2becff9640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 0x7f2bdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:32.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.923+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f2be80abc00 con 0x7f2bf00c6e00 2024-05-09T08:58:32.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.923+0000 7f2beffff640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 0x7f2bdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:32.924 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.923+0000 7f2beffff640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 0x7f2bdc06e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:32.926 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:32.927+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2be80b2080 con 0x7f2bf00c6e00 2024-05-09T08:58:33.105 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.106+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bf00cc970 con 0x7f2bdc06be30 2024-05-09T08:58:33.112 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.113+0000 7f2becff9640 1 -- 172.21.15.49:0/3497706170 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252343 (secure 0 0 0) 0x7f2bf00cc970 con 0x7f2bdc06be30 2024-05-09T08:58:33.119 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:33.120 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 msgr2=0x7f2bdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 0x7f2bdc06e2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 msgr2=0x7f2bf0157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2be8034040 tx=0x7f2be8004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 shutdown_connections 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2bdc06be30 0x7f2bdc06e2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2bf00c6e00 0x7f2bf0157910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bf00c66c0 0x7f2bf00cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.122 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 --2- 172.21.15.49:0/3497706170 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2bf0072300 0x7f2bf00ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 >> 172.21.15.49:0/3497706170 conn(0x7f2bf0074840 msgr2=0x7f2bf00c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 shutdown_connections 2024-05-09T08:58:33.123 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.116+0000 7f2bf62d4640 1 -- 172.21.15.49:0/3497706170 wait complete. 2024-05-09T08:58:33.127 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-05-09T08:58:32.602962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634649,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":406,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239544,"kb_used_data":9052,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735176,"statfs":{"total":773094113280,"available":772848820224,"internally_reserved":0,"allocated":9269248,"data_stored":4889600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226579"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598479+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:29.598479+0000","last_peered":"2024-05-09T08:58:29.598479+0000","last_clean":"2024-05-09T08:58:29.598479+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:29.598479+0000","last_undegraded":"2024-05-09T08:58:29.598479+0000","last_fullsized":"2024-05-09T08:58:29.598479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393481+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:30.393481+0000","last_peered":"2024-05-09T08:58:30.393481+0000","last_clean":"2024-05-09T08:58:30.393481+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:30.393481+0000","last_undegraded":"2024-05-09T08:58:30.393481+0000","last_fullsized":"2024-05-09T08:58:30.393481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395271+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:30.395271+0000","last_peered":"2024-05-09T08:58:30.395271+0000","last_clean":"2024-05-09T08:58:30.395271+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:30.395271+0000","last_undegraded":"2024-05-09T08:58:30.395271+0000","last_fullsized":"2024-05-09T08:58:30.395271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395664+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:30.395664+0000","last_peered":"2024-05-09T08:58:30.395664+0000","last_clean":"2024-05-09T08:58:30.395664+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:30.395664+0000","last_undegraded":"2024-05-09T08:58:30.395664+0000","last_fullsized":"2024-05-09T08:58:30.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395506+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395506+0000","last_peered":"2024-05-09T08:58:30.395506+0000","last_clean":"2024-05-09T08:58:30.395506+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395506+0000","last_undegraded":"2024-05-09T08:58:30.395506+0000","last_fullsized":"2024-05-09T08:58:30.395506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395310+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:30.395310+0000","last_peered":"2024-05-09T08:58:30.395310+0000","last_clean":"2024-05-09T08:58:30.395310+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:30.395310+0000","last_undegraded":"2024-05-09T08:58:30.395310+0000","last_fullsized":"2024-05-09T08:58:30.395310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396728+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396728+0000","last_peered":"2024-05-09T08:58:30.396728+0000","last_clean":"2024-05-09T08:58:30.396728+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396728+0000","last_undegraded":"2024-05-09T08:58:30.396728+0000","last_fullsized":"2024-05-09T08:58:30.396728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395275+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.395275+0000","last_peered":"2024-05-09T08:58:30.395275+0000","last_clean":"2024-05-09T08:58:30.395275+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.395275+0000","last_undegraded":"2024-05-09T08:58:30.395275+0000","last_fullsized":"2024-05-09T08:58:30.395275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394495+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:29.394495+0000","last_peered":"2024-05-09T08:58:29.394495+0000","last_clean":"2024-05-09T08:58:29.394495+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:29.394495+0000","last_undegraded":"2024-05-09T08:58:29.394495+0000","last_fullsized":"2024-05-09T08:58:29.394495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394484+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:29.394484+0000","last_peered":"2024-05-09T08:58:29.394484+0000","last_clean":"2024-05-09T08:58:29.394484+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:29.394484+0000","last_undegraded":"2024-05-09T08:58:29.394484+0000","last_fullsized":"2024-05-09T08:58:29.394484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395098+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395098+0000","last_peered":"2024-05-09T08:58:30.395098+0000","last_clean":"2024-05-09T08:58:30.395098+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395098+0000","last_undegraded":"2024-05-09T08:58:30.395098+0000","last_fullsized":"2024-05-09T08:58:30.395098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"63'3","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393249+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393249+0000","last_peered":"2024-05-09T08:58:30.393249+0000","last_clean":"2024-05-09T08:58:30.393249+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393249+0000","last_undegraded":"2024-05-09T08:58:30.393249+0000","last_fullsized":"2024-05-09T08:58:30.393249+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598225+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:29.598225+0000","last_peered":"2024-05-09T08:58:29.598225+0000","last_clean":"2024-05-09T08:58:29.598225+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:29.598225+0000","last_undegraded":"2024-05-09T08:58:29.598225+0000","last_fullsized":"2024-05-09T08:58:29.598225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396274+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396274+0000","last_peered":"2024-05-09T08:58:30.396274+0000","last_clean":"2024-05-09T08:58:30.396274+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396274+0000","last_undegraded":"2024-05-09T08:58:30.396274+0000","last_fullsized":"2024-05-09T08:58:30.396274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395329+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:30.395329+0000","last_peered":"2024-05-09T08:58:30.395329+0000","last_clean":"2024-05-09T08:58:30.395329+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:30.395329+0000","last_undegraded":"2024-05-09T08:58:30.395329+0000","last_fullsized":"2024-05-09T08:58:30.395329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394214+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:29.394214+0000","last_peered":"2024-05-09T08:58:29.394214+0000","last_clean":"2024-05-09T08:58:29.394214+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:29.394214+0000","last_undegraded":"2024-05-09T08:58:29.394214+0000","last_fullsized":"2024-05-09T08:58:29.394214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394550+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:29.394550+0000","last_peered":"2024-05-09T08:58:29.394550+0000","last_clean":"2024-05-09T08:58:29.394550+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:29.394550+0000","last_undegraded":"2024-05-09T08:58:29.394550+0000","last_fullsized":"2024-05-09T08:58:29.394550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598470+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:29.598470+0000","last_peered":"2024-05-09T08:58:29.598470+0000","last_clean":"2024-05-09T08:58:29.598470+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:29.598470+0000","last_undegraded":"2024-05-09T08:58:29.598470+0000","last_fullsized":"2024-05-09T08:58:29.598470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598367+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:29.598367+0000","last_peered":"2024-05-09T08:58:29.598367+0000","last_clean":"2024-05-09T08:58:29.598367+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:29.598367+0000","last_undegraded":"2024-05-09T08:58:29.598367+0000","last_fullsized":"2024-05-09T08:58:29.598367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393522+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:30.393522+0000","last_peered":"2024-05-09T08:58:30.393522+0000","last_clean":"2024-05-09T08:58:30.393522+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:30.393522+0000","last_undegraded":"2024-05-09T08:58:30.393522+0000","last_fullsized":"2024-05-09T08:58:30.393522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395699+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:30.395699+0000","last_peered":"2024-05-09T08:58:30.395699+0000","last_clean":"2024-05-09T08:58:30.395699+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:30.395699+0000","last_undegraded":"2024-05-09T08:58:30.395699+0000","last_fullsized":"2024-05-09T08:58:30.395699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395129+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:30.395129+0000","last_peered":"2024-05-09T08:58:30.395129+0000","last_clean":"2024-05-09T08:58:30.395129+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:30.395129+0000","last_undegraded":"2024-05-09T08:58:30.395129+0000","last_fullsized":"2024-05-09T08:58:30.395129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393980+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:29.393980+0000","last_peered":"2024-05-09T08:58:29.393980+0000","last_clean":"2024-05-09T08:58:29.393980+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:29.393980+0000","last_undegraded":"2024-05-09T08:58:29.393980+0000","last_fullsized":"2024-05-09T08:58:29.393980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394488+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:30.394488+0000","last_peered":"2024-05-09T08:58:30.394488+0000","last_clean":"2024-05-09T08:58:30.394488+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:30.394488+0000","last_undegraded":"2024-05-09T08:58:30.394488+0000","last_fullsized":"2024-05-09T08:58:30.394488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397388+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397388+0000","last_peered":"2024-05-09T08:58:30.397388+0000","last_clean":"2024-05-09T08:58:30.397388+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397388+0000","last_undegraded":"2024-05-09T08:58:30.397388+0000","last_fullsized":"2024-05-09T08:58:30.397388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393472+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393472+0000","last_peered":"2024-05-09T08:58:30.393472+0000","last_clean":"2024-05-09T08:58:30.393472+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393472+0000","last_undegraded":"2024-05-09T08:58:30.393472+0000","last_fullsized":"2024-05-09T08:58:30.393472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394161+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:29.394161+0000","last_peered":"2024-05-09T08:58:29.394161+0000","last_clean":"2024-05-09T08:58:29.394161+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:29.394161+0000","last_undegraded":"2024-05-09T08:58:29.394161+0000","last_fullsized":"2024-05-09T08:58:29.394161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396790+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:30.396790+0000","last_peered":"2024-05-09T08:58:30.396790+0000","last_clean":"2024-05-09T08:58:30.396790+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:30.396790+0000","last_undegraded":"2024-05-09T08:58:30.396790+0000","last_fullsized":"2024-05-09T08:58:30.396790+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598253+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:29.598253+0000","last_peered":"2024-05-09T08:58:29.598253+0000","last_clean":"2024-05-09T08:58:29.598253+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:29.598253+0000","last_undegraded":"2024-05-09T08:58:29.598253+0000","last_fullsized":"2024-05-09T08:58:29.598253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"63'6","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394549+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:29.394549+0000","last_peered":"2024-05-09T08:58:29.394549+0000","last_clean":"2024-05-09T08:58:29.394549+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:29.394549+0000","last_undegraded":"2024-05-09T08:58:29.394549+0000","last_fullsized":"2024-05-09T08:58:29.394549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395901+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395901+0000","last_peered":"2024-05-09T08:58:30.395901+0000","last_clean":"2024-05-09T08:58:30.395901+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395901+0000","last_undegraded":"2024-05-09T08:58:30.395901+0000","last_fullsized":"2024-05-09T08:58:30.395901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598427+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:29.598427+0000","last_peered":"2024-05-09T08:58:29.598427+0000","last_clean":"2024-05-09T08:58:29.598427+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:29.598427+0000","last_undegraded":"2024-05-09T08:58:29.598427+0000","last_fullsized":"2024-05-09T08:58:29.598427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"64'26","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394331+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:29.394331+0000","last_peered":"2024-05-09T08:58:29.394331+0000","last_clean":"2024-05-09T08:58:29.394331+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:29.394331+0000","last_undegraded":"2024-05-09T08:58:29.394331+0000","last_fullsized":"2024-05-09T08:58:29.394331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395845+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395845+0000","last_peered":"2024-05-09T08:58:30.395845+0000","last_clean":"2024-05-09T08:58:30.395845+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395845+0000","last_undegraded":"2024-05-09T08:58:30.395845+0000","last_fullsized":"2024-05-09T08:58:30.395845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395698+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395698+0000","last_peered":"2024-05-09T08:58:30.395698+0000","last_clean":"2024-05-09T08:58:30.395698+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395698+0000","last_undegraded":"2024-05-09T08:58:30.395698+0000","last_fullsized":"2024-05-09T08:58:30.395698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394169+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:29.394169+0000","last_peered":"2024-05-09T08:58:29.394169+0000","last_clean":"2024-05-09T08:58:29.394169+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:29.394169+0000","last_undegraded":"2024-05-09T08:58:29.394169+0000","last_fullsized":"2024-05-09T08:58:29.394169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394298+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:29.394298+0000","last_peered":"2024-05-09T08:58:29.394298+0000","last_clean":"2024-05-09T08:58:29.394298+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:29.394298+0000","last_undegraded":"2024-05-09T08:58:29.394298+0000","last_fullsized":"2024-05-09T08:58:29.394298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598545+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:29.598545+0000","last_peered":"2024-05-09T08:58:29.598545+0000","last_clean":"2024-05-09T08:58:29.598545+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:29.598545+0000","last_undegraded":"2024-05-09T08:58:29.598545+0000","last_fullsized":"2024-05-09T08:58:29.598545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598234+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:29.598234+0000","last_peered":"2024-05-09T08:58:29.598234+0000","last_clean":"2024-05-09T08:58:29.598234+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:29.598234+0000","last_undegraded":"2024-05-09T08:58:29.598234+0000","last_fullsized":"2024-05-09T08:58:29.598234+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394906+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:30.394906+0000","last_peered":"2024-05-09T08:58:30.394906+0000","last_clean":"2024-05-09T08:58:30.394906+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:30.394906+0000","last_undegraded":"2024-05-09T08:58:30.394906+0000","last_fullsized":"2024-05-09T08:58:30.394906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598472+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:29.598472+0000","last_peered":"2024-05-09T08:58:29.598472+0000","last_clean":"2024-05-09T08:58:29.598472+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:29.598472+0000","last_undegraded":"2024-05-09T08:58:29.598472+0000","last_fullsized":"2024-05-09T08:58:29.598472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396529+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396529+0000","last_peered":"2024-05-09T08:58:30.396529+0000","last_clean":"2024-05-09T08:58:30.396529+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396529+0000","last_undegraded":"2024-05-09T08:58:30.396529+0000","last_fullsized":"2024-05-09T08:58:30.396529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598370+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:29.598370+0000","last_peered":"2024-05-09T08:58:29.598370+0000","last_clean":"2024-05-09T08:58:29.598370+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:29.598370+0000","last_undegraded":"2024-05-09T08:58:29.598370+0000","last_fullsized":"2024-05-09T08:58:29.598370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598589+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:29.598589+0000","last_peered":"2024-05-09T08:58:29.598589+0000","last_clean":"2024-05-09T08:58:29.598589+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:29.598589+0000","last_undegraded":"2024-05-09T08:58:29.598589+0000","last_fullsized":"2024-05-09T08:58:29.598589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397204+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397204+0000","last_peered":"2024-05-09T08:58:30.397204+0000","last_clean":"2024-05-09T08:58:30.397204+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397204+0000","last_undegraded":"2024-05-09T08:58:30.397204+0000","last_fullsized":"2024-05-09T08:58:30.397204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394674+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:29.394674+0000","last_peered":"2024-05-09T08:58:29.394674+0000","last_clean":"2024-05-09T08:58:29.394674+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:29.394674+0000","last_undegraded":"2024-05-09T08:58:29.394674+0000","last_fullsized":"2024-05-09T08:58:29.394674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394637+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394637+0000","last_peered":"2024-05-09T08:58:30.394637+0000","last_clean":"2024-05-09T08:58:30.394637+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394637+0000","last_undegraded":"2024-05-09T08:58:30.394637+0000","last_fullsized":"2024-05-09T08:58:30.394637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598317+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:29.598317+0000","last_peered":"2024-05-09T08:58:29.598317+0000","last_clean":"2024-05-09T08:58:29.598317+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:29.598317+0000","last_undegraded":"2024-05-09T08:58:29.598317+0000","last_fullsized":"2024-05-09T08:58:29.598317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396900+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396900+0000","last_peered":"2024-05-09T08:58:30.396900+0000","last_clean":"2024-05-09T08:58:30.396900+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396900+0000","last_undegraded":"2024-05-09T08:58:30.396900+0000","last_fullsized":"2024-05-09T08:58:30.396900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598366+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:29.598366+0000","last_peered":"2024-05-09T08:58:29.598366+0000","last_clean":"2024-05-09T08:58:29.598366+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:29.598366+0000","last_undegraded":"2024-05-09T08:58:29.598366+0000","last_fullsized":"2024-05-09T08:58:29.598366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397263+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:30.397263+0000","last_peered":"2024-05-09T08:58:30.397263+0000","last_clean":"2024-05-09T08:58:30.397263+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:30.397263+0000","last_undegraded":"2024-05-09T08:58:30.397263+0000","last_fullsized":"2024-05-09T08:58:30.397263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393999+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:29.393999+0000","last_peered":"2024-05-09T08:58:29.393999+0000","last_clean":"2024-05-09T08:58:29.393999+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:29.393999+0000","last_undegraded":"2024-05-09T08:58:29.393999+0000","last_fullsized":"2024-05-09T08:58:29.393999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395241+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:30.395241+0000","last_peered":"2024-05-09T08:58:30.395241+0000","last_clean":"2024-05-09T08:58:30.395241+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:30.395241+0000","last_undegraded":"2024-05-09T08:58:30.395241+0000","last_fullsized":"2024-05-09T08:58:30.395241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395554+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395554+0000","last_peered":"2024-05-09T08:58:30.395554+0000","last_clean":"2024-05-09T08:58:30.395554+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395554+0000","last_undegraded":"2024-05-09T08:58:30.395554+0000","last_fullsized":"2024-05-09T08:58:30.395554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44281,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":88562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28588,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343252,"statfs":{"total":96636764160,"available":96607490048,"internally_reserved":0,"allocated":1597440,"data_stored":1044500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1028096,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":455708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":453106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":490459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874246,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29808,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342032,"statfs":{"total":96636764160,"available":96606240768,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":453308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1043018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:33.131 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:33.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.272+0000 7ff28d438640 1 Processor -- start 2024-05-09T08:58:33.272 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.272+0000 7ff28d438640 1 -- start start 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff28d438640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880c9c00 0x7ff2880c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff28d438640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2880c8350 con 0x7ff288151e00 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff28d438640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff288158490 con 0x7ff2880c8780 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff28d438640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff2880c7510 con 0x7ff2880c9c00 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff286ffd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7ff288151e00 0x7ff2880c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49652/0 (socket says 172.21.15.49:49652) 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff286ffd640 1 -- 172.21.15.49:0/4096883387 learned_addr learned my addr 172.21.15.49:0/4096883387 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:33.274 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1411994171 0 0) 0x7ff280001490 con 0x7ff288151e00 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff2867fc640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880c9c00 0x7ff2880c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.273+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff274005730 con 0x7ff288151e00 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2077784725 0 0) 0x7ff274005730 con 0x7ff288151e00 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880c9c00 msgr2=0x7ff2880c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff2880c9c00 0x7ff2880c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 >> v1:172.21.15.183:6789/0 conn(0x7ff2880c8780 legacy=0x7ff2881622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff288158750 con 0x7ff288151e00 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7ff280001bc0 con 0x7ff288151e00 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 >> v1:172.21.15.49:6789/0 conn(0x7ff288151e00 legacy=0x7ff2880c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff274005bf0 0x7ff274006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274006a10 0x7ff274006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.275 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff274007390 0x7ff27400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880c7510 con 0x7ff274005bf0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff27400e110 con 0x7ff274007390 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff27400e2c0 con 0x7ff274006a10 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff274005bf0 0x7ff274006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff2867fc640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274006a10 0x7ff274006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff2877fe640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff274007390 0x7ff27400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 -- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274006a10 msgr2=0x7ff274006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274006a10 0x7ff274006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 -- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff274007390 msgr2=0x7ff27400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff274007390 0x7ff27400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.274+0000 7ff286ffd640 1 -- 172.21.15.49:0/4096883387 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274005730 con 0x7ff274005bf0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.275+0000 7ff286ffd640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff274005bf0 0x7ff274006010 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff28000aa50 tx=0x7ff274007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16202b745b898586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.275+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000a670 con 0x7ff274005bf0 2024-05-09T08:58:33.276 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.275+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28002f070 con 0x7ff274005bf0 2024-05-09T08:58:33.277 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.275+0000 7ff285ffb640 1 -- 172.21.15.49:0/4096883387 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280034470 con 0x7ff274005bf0 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 -- 172.21.15.49:0/4096883387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff274005bf0 msgr2=0x7ff274006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 --2- 172.21.15.49:0/4096883387 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff274005bf0 0x7ff274006010 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff28000aa50 tx=0x7ff274007a40 comp rx=0 tx=0).stop 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff2877fe640 1 -- 172.21.15.49:0/4096883387 reap_dead start 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 -- 172.21.15.49:0/4096883387 shutdown_connections 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 -- 172.21.15.49:0/4096883387 >> 172.21.15.49:0/4096883387 conn(0x7ff288074840 msgr2=0x7ff288074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 -- 172.21.15.49:0/4096883387 shutdown_connections 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.276+0000 7ff28d438640 1 -- 172.21.15.49:0/4096883387 wait complete. 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.277+0000 7ff28d438640 1 Processor -- start 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.277+0000 7ff28d438640 1 -- start start 2024-05-09T08:58:33.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274005bf0 0x7ff28815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 0x7ff288158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880718d0 con 0x7ff2881f0c80 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881521e0 con 0x7ff288157ac0 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff28d438640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7ff288071bb0 con 0x7ff274005bf0 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52680/0 (socket says 172.21.15.49:52680) 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 -- 172.21.15.49:0/682804157 learned_addr learned my addr 172.21.15.49:0/682804157 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:33.281 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff286ffd640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274005bf0 0x7ff28815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2867fc640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 0x7ff288158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 -- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274005bf0 msgr2=0x7ff28815b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274005bf0 0x7ff28815b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 -- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 msgr2=0x7ff288158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 0x7ff288158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 -- 172.21.15.49:0/682804157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881f1640 con 0x7ff2881f0c80 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2867fc640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 0x7ff288158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff2877fe640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cd4c0 tx=0x7ff27800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780053a0 con 0x7ff2881f0c80 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff278018070 con 0x7ff2881f0c80 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.278+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278003830 con 0x7ff2881f0c80 2024-05-09T08:58:33.282 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.279+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881f3e90 con 0x7ff2881f0c80 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.279+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881f4450 con 0x7ff2881f0c80 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.280+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff288075850 con 0x7ff2881f0c80 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.280+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2780039d0 con 0x7ff2881f0c80 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.280+0000 7ff27f7fe640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 0x7ff26806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.281+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7ff27808ab10 con 0x7ff2881f0c80 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.281+0000 7ff286ffd640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 0x7ff26806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.281+0000 7ff286ffd640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 0x7ff26806e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff28000aa20 tx=0x7ff280009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.285+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff288075850 con 0x7ff2881f0c80 2024-05-09T08:58:33.463 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.464+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff288068560 con 0x7ff26806be60 2024-05-09T08:58:33.470 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.471+0000 7ff27f7fe640 1 -- 172.21.15.49:0/682804157 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252343 (secure 0 0 0) 0x7ff288068560 con 0x7ff26806be60 2024-05-09T08:58:33.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:33.474 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 msgr2=0x7ff26806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 0x7ff26806e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff28000aa20 tx=0x7ff280009030 comp rx=0 tx=0).stop 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 msgr2=0x7ff2881f1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2880cd4c0 tx=0x7ff27800f920 comp rx=0 tx=0).stop 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 shutdown_connections 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7ff26806be60 0x7ff26806e320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7ff2881f0c80 0x7ff2881f1100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff288157ac0 0x7ff288158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.476 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 --2- 172.21.15.49:0/682804157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff274005bf0 0x7ff28815b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 >> 172.21.15.49:0/682804157 conn(0x7ff288074840 msgr2=0x7ff2880c4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 shutdown_connections 2024-05-09T08:58:33.477 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.474+0000 7ff28d438640 1 -- 172.21.15.49:0/682804157 wait complete. 2024-05-09T08:58:33.485 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-05-09T08:58:32.602962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634649,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":406,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239544,"kb_used_data":9052,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735176,"statfs":{"total":773094113280,"available":772848820224,"internally_reserved":0,"allocated":9269248,"data_stored":4889600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.226579"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598479+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:29.598479+0000","last_peered":"2024-05-09T08:58:29.598479+0000","last_clean":"2024-05-09T08:58:29.598479+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:29.598479+0000","last_undegraded":"2024-05-09T08:58:29.598479+0000","last_fullsized":"2024-05-09T08:58:29.598479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393481+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:30.393481+0000","last_peered":"2024-05-09T08:58:30.393481+0000","last_clean":"2024-05-09T08:58:30.393481+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:30.393481+0000","last_undegraded":"2024-05-09T08:58:30.393481+0000","last_fullsized":"2024-05-09T08:58:30.393481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395271+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:30.395271+0000","last_peered":"2024-05-09T08:58:30.395271+0000","last_clean":"2024-05-09T08:58:30.395271+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:30.395271+0000","last_undegraded":"2024-05-09T08:58:30.395271+0000","last_fullsized":"2024-05-09T08:58:30.395271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395664+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:30.395664+0000","last_peered":"2024-05-09T08:58:30.395664+0000","last_clean":"2024-05-09T08:58:30.395664+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:30.395664+0000","last_undegraded":"2024-05-09T08:58:30.395664+0000","last_fullsized":"2024-05-09T08:58:30.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395506+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395506+0000","last_peered":"2024-05-09T08:58:30.395506+0000","last_clean":"2024-05-09T08:58:30.395506+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395506+0000","last_undegraded":"2024-05-09T08:58:30.395506+0000","last_fullsized":"2024-05-09T08:58:30.395506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395310+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:30.395310+0000","last_peered":"2024-05-09T08:58:30.395310+0000","last_clean":"2024-05-09T08:58:30.395310+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:30.395310+0000","last_undegraded":"2024-05-09T08:58:30.395310+0000","last_fullsized":"2024-05-09T08:58:30.395310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396728+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396728+0000","last_peered":"2024-05-09T08:58:30.396728+0000","last_clean":"2024-05-09T08:58:30.396728+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396728+0000","last_undegraded":"2024-05-09T08:58:30.396728+0000","last_fullsized":"2024-05-09T08:58:30.396728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395275+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.395275+0000","last_peered":"2024-05-09T08:58:30.395275+0000","last_clean":"2024-05-09T08:58:30.395275+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.395275+0000","last_undegraded":"2024-05-09T08:58:30.395275+0000","last_fullsized":"2024-05-09T08:58:30.395275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394495+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:29.394495+0000","last_peered":"2024-05-09T08:58:29.394495+0000","last_clean":"2024-05-09T08:58:29.394495+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:29.394495+0000","last_undegraded":"2024-05-09T08:58:29.394495+0000","last_fullsized":"2024-05-09T08:58:29.394495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394484+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:29.394484+0000","last_peered":"2024-05-09T08:58:29.394484+0000","last_clean":"2024-05-09T08:58:29.394484+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:29.394484+0000","last_undegraded":"2024-05-09T08:58:29.394484+0000","last_fullsized":"2024-05-09T08:58:29.394484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395098+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395098+0000","last_peered":"2024-05-09T08:58:30.395098+0000","last_clean":"2024-05-09T08:58:30.395098+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395098+0000","last_undegraded":"2024-05-09T08:58:30.395098+0000","last_fullsized":"2024-05-09T08:58:30.395098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"63'3","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393249+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393249+0000","last_peered":"2024-05-09T08:58:30.393249+0000","last_clean":"2024-05-09T08:58:30.393249+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393249+0000","last_undegraded":"2024-05-09T08:58:30.393249+0000","last_fullsized":"2024-05-09T08:58:30.393249+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598225+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:29.598225+0000","last_peered":"2024-05-09T08:58:29.598225+0000","last_clean":"2024-05-09T08:58:29.598225+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:29.598225+0000","last_undegraded":"2024-05-09T08:58:29.598225+0000","last_fullsized":"2024-05-09T08:58:29.598225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396274+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396274+0000","last_peered":"2024-05-09T08:58:30.396274+0000","last_clean":"2024-05-09T08:58:30.396274+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396274+0000","last_undegraded":"2024-05-09T08:58:30.396274+0000","last_fullsized":"2024-05-09T08:58:30.396274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395329+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:30.395329+0000","last_peered":"2024-05-09T08:58:30.395329+0000","last_clean":"2024-05-09T08:58:30.395329+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:30.395329+0000","last_undegraded":"2024-05-09T08:58:30.395329+0000","last_fullsized":"2024-05-09T08:58:30.395329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394214+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:29.394214+0000","last_peered":"2024-05-09T08:58:29.394214+0000","last_clean":"2024-05-09T08:58:29.394214+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:29.394214+0000","last_undegraded":"2024-05-09T08:58:29.394214+0000","last_fullsized":"2024-05-09T08:58:29.394214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394550+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:29.394550+0000","last_peered":"2024-05-09T08:58:29.394550+0000","last_clean":"2024-05-09T08:58:29.394550+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:29.394550+0000","last_undegraded":"2024-05-09T08:58:29.394550+0000","last_fullsized":"2024-05-09T08:58:29.394550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598470+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:29.598470+0000","last_peered":"2024-05-09T08:58:29.598470+0000","last_clean":"2024-05-09T08:58:29.598470+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:29.598470+0000","last_undegraded":"2024-05-09T08:58:29.598470+0000","last_fullsized":"2024-05-09T08:58:29.598470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598367+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:29.598367+0000","last_peered":"2024-05-09T08:58:29.598367+0000","last_clean":"2024-05-09T08:58:29.598367+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:29.598367+0000","last_undegraded":"2024-05-09T08:58:29.598367+0000","last_fullsized":"2024-05-09T08:58:29.598367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393522+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:30.393522+0000","last_peered":"2024-05-09T08:58:30.393522+0000","last_clean":"2024-05-09T08:58:30.393522+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:30.393522+0000","last_undegraded":"2024-05-09T08:58:30.393522+0000","last_fullsized":"2024-05-09T08:58:30.393522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395699+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:30.395699+0000","last_peered":"2024-05-09T08:58:30.395699+0000","last_clean":"2024-05-09T08:58:30.395699+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:30.395699+0000","last_undegraded":"2024-05-09T08:58:30.395699+0000","last_fullsized":"2024-05-09T08:58:30.395699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395129+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:30.395129+0000","last_peered":"2024-05-09T08:58:30.395129+0000","last_clean":"2024-05-09T08:58:30.395129+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:30.395129+0000","last_undegraded":"2024-05-09T08:58:30.395129+0000","last_fullsized":"2024-05-09T08:58:30.395129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393980+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:29.393980+0000","last_peered":"2024-05-09T08:58:29.393980+0000","last_clean":"2024-05-09T08:58:29.393980+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:29.393980+0000","last_undegraded":"2024-05-09T08:58:29.393980+0000","last_fullsized":"2024-05-09T08:58:29.393980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394488+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:30.394488+0000","last_peered":"2024-05-09T08:58:30.394488+0000","last_clean":"2024-05-09T08:58:30.394488+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:30.394488+0000","last_undegraded":"2024-05-09T08:58:30.394488+0000","last_fullsized":"2024-05-09T08:58:30.394488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397388+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397388+0000","last_peered":"2024-05-09T08:58:30.397388+0000","last_clean":"2024-05-09T08:58:30.397388+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397388+0000","last_undegraded":"2024-05-09T08:58:30.397388+0000","last_fullsized":"2024-05-09T08:58:30.397388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393472+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393472+0000","last_peered":"2024-05-09T08:58:30.393472+0000","last_clean":"2024-05-09T08:58:30.393472+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393472+0000","last_undegraded":"2024-05-09T08:58:30.393472+0000","last_fullsized":"2024-05-09T08:58:30.393472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394161+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:29.394161+0000","last_peered":"2024-05-09T08:58:29.394161+0000","last_clean":"2024-05-09T08:58:29.394161+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:29.394161+0000","last_undegraded":"2024-05-09T08:58:29.394161+0000","last_fullsized":"2024-05-09T08:58:29.394161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396790+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:30.396790+0000","last_peered":"2024-05-09T08:58:30.396790+0000","last_clean":"2024-05-09T08:58:30.396790+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:30.396790+0000","last_undegraded":"2024-05-09T08:58:30.396790+0000","last_fullsized":"2024-05-09T08:58:30.396790+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598253+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:29.598253+0000","last_peered":"2024-05-09T08:58:29.598253+0000","last_clean":"2024-05-09T08:58:29.598253+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:29.598253+0000","last_undegraded":"2024-05-09T08:58:29.598253+0000","last_fullsized":"2024-05-09T08:58:29.598253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"63'6","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394549+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:29.394549+0000","last_peered":"2024-05-09T08:58:29.394549+0000","last_clean":"2024-05-09T08:58:29.394549+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:29.394549+0000","last_undegraded":"2024-05-09T08:58:29.394549+0000","last_fullsized":"2024-05-09T08:58:29.394549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395901+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395901+0000","last_peered":"2024-05-09T08:58:30.395901+0000","last_clean":"2024-05-09T08:58:30.395901+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395901+0000","last_undegraded":"2024-05-09T08:58:30.395901+0000","last_fullsized":"2024-05-09T08:58:30.395901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598427+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:29.598427+0000","last_peered":"2024-05-09T08:58:29.598427+0000","last_clean":"2024-05-09T08:58:29.598427+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:29.598427+0000","last_undegraded":"2024-05-09T08:58:29.598427+0000","last_fullsized":"2024-05-09T08:58:29.598427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"64'26","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394331+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:29.394331+0000","last_peered":"2024-05-09T08:58:29.394331+0000","last_clean":"2024-05-09T08:58:29.394331+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:29.394331+0000","last_undegraded":"2024-05-09T08:58:29.394331+0000","last_fullsized":"2024-05-09T08:58:29.394331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395845+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395845+0000","last_peered":"2024-05-09T08:58:30.395845+0000","last_clean":"2024-05-09T08:58:30.395845+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395845+0000","last_undegraded":"2024-05-09T08:58:30.395845+0000","last_fullsized":"2024-05-09T08:58:30.395845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395698+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395698+0000","last_peered":"2024-05-09T08:58:30.395698+0000","last_clean":"2024-05-09T08:58:30.395698+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395698+0000","last_undegraded":"2024-05-09T08:58:30.395698+0000","last_fullsized":"2024-05-09T08:58:30.395698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394169+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:29.394169+0000","last_peered":"2024-05-09T08:58:29.394169+0000","last_clean":"2024-05-09T08:58:29.394169+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:29.394169+0000","last_undegraded":"2024-05-09T08:58:29.394169+0000","last_fullsized":"2024-05-09T08:58:29.394169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394298+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:29.394298+0000","last_peered":"2024-05-09T08:58:29.394298+0000","last_clean":"2024-05-09T08:58:29.394298+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:29.394298+0000","last_undegraded":"2024-05-09T08:58:29.394298+0000","last_fullsized":"2024-05-09T08:58:29.394298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598545+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:29.598545+0000","last_peered":"2024-05-09T08:58:29.598545+0000","last_clean":"2024-05-09T08:58:29.598545+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:29.598545+0000","last_undegraded":"2024-05-09T08:58:29.598545+0000","last_fullsized":"2024-05-09T08:58:29.598545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598234+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:29.598234+0000","last_peered":"2024-05-09T08:58:29.598234+0000","last_clean":"2024-05-09T08:58:29.598234+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:29.598234+0000","last_undegraded":"2024-05-09T08:58:29.598234+0000","last_fullsized":"2024-05-09T08:58:29.598234+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394906+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:30.394906+0000","last_peered":"2024-05-09T08:58:30.394906+0000","last_clean":"2024-05-09T08:58:30.394906+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:30.394906+0000","last_undegraded":"2024-05-09T08:58:30.394906+0000","last_fullsized":"2024-05-09T08:58:30.394906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598472+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:29.598472+0000","last_peered":"2024-05-09T08:58:29.598472+0000","last_clean":"2024-05-09T08:58:29.598472+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:29.598472+0000","last_undegraded":"2024-05-09T08:58:29.598472+0000","last_fullsized":"2024-05-09T08:58:29.598472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396529+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396529+0000","last_peered":"2024-05-09T08:58:30.396529+0000","last_clean":"2024-05-09T08:58:30.396529+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396529+0000","last_undegraded":"2024-05-09T08:58:30.396529+0000","last_fullsized":"2024-05-09T08:58:30.396529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598370+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:29.598370+0000","last_peered":"2024-05-09T08:58:29.598370+0000","last_clean":"2024-05-09T08:58:29.598370+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:29.598370+0000","last_undegraded":"2024-05-09T08:58:29.598370+0000","last_fullsized":"2024-05-09T08:58:29.598370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598589+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:29.598589+0000","last_peered":"2024-05-09T08:58:29.598589+0000","last_clean":"2024-05-09T08:58:29.598589+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:29.598589+0000","last_undegraded":"2024-05-09T08:58:29.598589+0000","last_fullsized":"2024-05-09T08:58:29.598589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397204+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397204+0000","last_peered":"2024-05-09T08:58:30.397204+0000","last_clean":"2024-05-09T08:58:30.397204+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397204+0000","last_undegraded":"2024-05-09T08:58:30.397204+0000","last_fullsized":"2024-05-09T08:58:30.397204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394674+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:29.394674+0000","last_peered":"2024-05-09T08:58:29.394674+0000","last_clean":"2024-05-09T08:58:29.394674+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:29.394674+0000","last_undegraded":"2024-05-09T08:58:29.394674+0000","last_fullsized":"2024-05-09T08:58:29.394674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394637+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394637+0000","last_peered":"2024-05-09T08:58:30.394637+0000","last_clean":"2024-05-09T08:58:30.394637+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394637+0000","last_undegraded":"2024-05-09T08:58:30.394637+0000","last_fullsized":"2024-05-09T08:58:30.394637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598317+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:29.598317+0000","last_peered":"2024-05-09T08:58:29.598317+0000","last_clean":"2024-05-09T08:58:29.598317+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:29.598317+0000","last_undegraded":"2024-05-09T08:58:29.598317+0000","last_fullsized":"2024-05-09T08:58:29.598317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396900+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396900+0000","last_peered":"2024-05-09T08:58:30.396900+0000","last_clean":"2024-05-09T08:58:30.396900+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396900+0000","last_undegraded":"2024-05-09T08:58:30.396900+0000","last_fullsized":"2024-05-09T08:58:30.396900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598366+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:29.598366+0000","last_peered":"2024-05-09T08:58:29.598366+0000","last_clean":"2024-05-09T08:58:29.598366+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:29.598366+0000","last_undegraded":"2024-05-09T08:58:29.598366+0000","last_fullsized":"2024-05-09T08:58:29.598366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397263+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:30.397263+0000","last_peered":"2024-05-09T08:58:30.397263+0000","last_clean":"2024-05-09T08:58:30.397263+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:30.397263+0000","last_undegraded":"2024-05-09T08:58:30.397263+0000","last_fullsized":"2024-05-09T08:58:30.397263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393999+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:29.393999+0000","last_peered":"2024-05-09T08:58:29.393999+0000","last_clean":"2024-05-09T08:58:29.393999+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:29.393999+0000","last_undegraded":"2024-05-09T08:58:29.393999+0000","last_fullsized":"2024-05-09T08:58:29.393999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395241+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:30.395241+0000","last_peered":"2024-05-09T08:58:30.395241+0000","last_clean":"2024-05-09T08:58:30.395241+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:30.395241+0000","last_undegraded":"2024-05-09T08:58:30.395241+0000","last_fullsized":"2024-05-09T08:58:30.395241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395554+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395554+0000","last_peered":"2024-05-09T08:58:30.395554+0000","last_clean":"2024-05-09T08:58:30.395554+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395554+0000","last_undegraded":"2024-05-09T08:58:30.395554+0000","last_fullsized":"2024-05-09T08:58:30.395554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44281,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":88562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28588,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343252,"statfs":{"total":96636764160,"available":96607490048,"internally_reserved":0,"allocated":1597440,"data_stored":1044500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1028096,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":455708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":453106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":490459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874246,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29808,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342032,"statfs":{"total":96636764160,"available":96606240768,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":453308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1043018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:33.489 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-09T08:58:33.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-09T08:58:33.630 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.631+0000 7f74fe0b3640 1 Processor -- start 2024-05-09T08:58:33.631 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.631+0000 7f74fe0b3640 1 -- start start 2024-05-09T08:58:33.632 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74fe0b3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74fe0b3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74f8072670 0x7f74f80c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74fe0b3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f8159b10 con 0x7f74f8152e30 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74fe0b3640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f74f8159270 con 0x7f74f8072670 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74fe0b3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74f8159550 con 0x7f74f80c7130 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74f7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74f8072670 0x7f74f80c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74f7fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74f8072670 0x7f74f80c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52682/0 (socket says 172.21.15.49:52682) 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.632+0000 7f74f7fff640 1 -- 172.21.15.49:0/2228188612 learned_addr learned my addr 172.21.15.49:0/2228188612 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f7fff640 1 -- 172.21.15.49:0/2228188612 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 msgr2=0x7f74f8158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f7fff640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f8158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f7fff640 1 -- 172.21.15.49:0/2228188612 >> v1:172.21.15.183:6789/0 conn(0x7f74f8152e30 legacy=0x7f74f80c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:33.633 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f7fff640 1 -- 172.21.15.49:0/2228188612 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8159830 con 0x7f74f8072670 2024-05-09T08:58:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.633+0000 7f74f7fff640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f80c3790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009e50 tx=0x7f74f002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5138489d626bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74f67fc640 1 -- 172.21.15.49:0/2228188612 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0036070 con 0x7f74f8072670 2024-05-09T08:58:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74f67fc640 1 -- 172.21.15.49:0/2228188612 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f0031600 con 0x7f74f8072670 2024-05-09T08:58:33.634 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74f67fc640 1 -- 172.21.15.49:0/2228188612 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f00319d0 con 0x7f74f8072670 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 -- 172.21.15.49:0/2228188612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 msgr2=0x7f74f80c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f80c3790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009e50 tx=0x7f74f002a6c0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 -- 172.21.15.49:0/2228188612 shutdown_connections 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f80c3790 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/2228188612 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 -- 172.21.15.49:0/2228188612 >> 172.21.15.49:0/2228188612 conn(0x7f74f80cc290 msgr2=0x7f74f80cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 -- 172.21.15.49:0/2228188612 shutdown_connections 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.634+0000 7f74fe0b3640 1 -- 172.21.15.49:0/2228188612 wait complete. 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.635+0000 7f74fe0b3640 1 Processor -- start 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 -- start start 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.637 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f81579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74f8152e30 0x7f74f81f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f74f80c6620 con 0x7f74f8072670 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74f80c59f0 con 0x7f74f8152e30 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74fe0b3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74f806b4b0 con 0x7f74f80c7130 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74f77fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52692/0 (socket says 172.21.15.49:52692) 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.636+0000 7f74f77fe640 1 -- 172.21.15.49:0/4124936920 learned_addr learned my addr 172.21.15.49:0/4124936920 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f7fff640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74f8152e30 0x7f74f81f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f81579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 -- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 msgr2=0x7f74f81579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f81579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 -- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74f8152e30 msgr2=0x7f74f81f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.638 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74f8152e30 0x7f74f81f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 -- 172.21.15.49:0/4124936920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f80c2fd0 con 0x7f74f8072670 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f81579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74f77fe640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b3f0 tx=0x7f74e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015020 con 0x7f74f8072670 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e8011830 con 0x7f74f8072670 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8011c20 con 0x7f74f8072670 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f8155150 con 0x7f74f8072670 2024-05-09T08:58:33.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.637+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f8155710 con 0x7f74f8072670 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.638+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f804f660 con 0x7f74f8072670 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.639+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f74e8018020 con 0x7f74f8072670 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.639+0000 7f74dffff640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 0x7f74e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.639+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f74e808aaf0 con 0x7f74f8072670 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.639+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 0x7f74e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:33.640 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.640+0000 7f74f6ffd640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 0x7f74e406e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006fd0 tx=0x7f74ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:33.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.643+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f74e801f030 con 0x7f74f8072670 2024-05-09T08:58:33.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.828+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7f74f80c3470 con 0x7f74f8072670 2024-05-09T08:58:33.830 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.831+0000 7f74dffff640 1 -- 172.21.15.49:0/4124936920 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7f74e80172c0 con 0x7f74f8072670 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 msgr2=0x7f74e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 0x7f74e406e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f74ec006fd0 tx=0x7f74ec008040 comp rx=0 tx=0).stop 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 msgr2=0x7f74f81e3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b3f0 tx=0x7f74e800b8b0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 shutdown_connections 2024-05-09T08:58:33.833 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f74e406be30 0x7f74e406e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74f8152e30 0x7f74f81f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74f80c7130 0x7f74f81579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 --2- 172.21.15.49:0/4124936920 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74f8072670 0x7f74f81e3a80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 >> 172.21.15.49:0/4124936920 conn(0x7f74f80cc290 msgr2=0x7f74f80c9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 shutdown_connections 2024-05-09T08:58:33.834 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:33.834+0000 7f74fe0b3640 1 -- 172.21.15.49:0/4124936920 wait complete. 2024-05-09T08:58:33.844 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T08:58:33.854 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T08:58:33.854 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:58:33.854 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:58:33.871 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-05-09T08:58:33.871 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T08:58:33.927 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:58:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.071+0000 7f8d63190640 1 Processor -- start 2024-05-09T08:58:34.071 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d63190640 1 -- start start 2024-05-09T08:58:34.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d63190640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d5c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d63190640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c15e260 con 0x7f8d5c1517f0 2024-05-09T08:58:34.074 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d63190640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5c15e520 con 0x7f8d5c0cc190 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d63190640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c151bd0 con 0x7f8d5c075200 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.072+0000 7f8d60f2c640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8d5c1517f0 0x7f8d5c0ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49662/0 (socket says 172.21.15.49:49662) 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d60f2c640 1 -- 172.21.15.49:0/2540489620 learned_addr learned my addr 172.21.15.49:0/2540489620 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5bfff640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d5c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955422501 0 0) 0x7f8d5c15e260 con 0x7f8d5c1517f0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d48003520 con 0x7f8d5c1517f0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2364770769 0 0) 0x7f8d48003520 con 0x7f8d5c1517f0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 msgr2=0x7f8d5c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d5c15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 >> v1:172.21.15.183:6789/0 conn(0x7f8d5c0cc190 legacy=0x7f8d5c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c15e800 con 0x7f8d5c1517f0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f8d54001670 con 0x7f8d5c1517f0 2024-05-09T08:58:34.075 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 >> v1:172.21.15.49:6789/0 conn(0x7f8d5c1517f0 legacy=0x7f8d5c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d480039e0 0x7f8d48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d48004800 0x7f8d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c151bd0 con 0x7f8d480039e0 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4800bb00 con 0x7f8d48004800 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4800bd00 con 0x7f8d5c075200 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d480039e0 0x7f8d48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d5bfff640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d48004800 0x7f8d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.073+0000 7f8d6172d640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 -- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 msgr2=0x7f8d480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c075200 0x7f8d480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 -- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d48004800 msgr2=0x7f8d48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.076 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d48004800 0x7f8d48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 -- 172.21.15.49:0/2540489620 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d48003520 con 0x7f8d480039e0 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d480039e0 0x7f8d48003e00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400c210 tx=0x7f8d48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c264948fe1c0a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54007cf0 con 0x7f8d480039e0 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d54005ad0 con 0x7f8d480039e0 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.074+0000 7f8d5b7fe640 1 -- 172.21.15.49:0/2540489620 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d54007070 con 0x7f8d480039e0 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 -- 172.21.15.49:0/2540489620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d480039e0 msgr2=0x7f8d48003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 --2- 172.21.15.49:0/2540489620 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d480039e0 0x7f8d48003e00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400c210 tx=0x7f8d48005830 comp rx=0 tx=0).stop 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d6172d640 1 -- 172.21.15.49:0/2540489620 reap_dead start 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 -- 172.21.15.49:0/2540489620 shutdown_connections 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 -- 172.21.15.49:0/2540489620 >> 172.21.15.49:0/2540489620 conn(0x7f8d5c06bd00 msgr2=0x7f8d5c074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 -- 172.21.15.49:0/2540489620 shutdown_connections 2024-05-09T08:58:34.077 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.075+0000 7f8d63190640 1 -- 172.21.15.49:0/2540489620 wait complete. 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.076+0000 7f8d63190640 1 Processor -- start 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.076+0000 7f8d63190640 1 -- start start 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d480039e0 0x7f8d5c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0d0180 0x7f8d5c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c15f0a0 con 0x7f8d5c14a810 2024-05-09T08:58:34.080 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c15edc0 con 0x7f8d480039e0 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d63190640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f8d5c0cbc70 con 0x7f8d5c0d0180 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d6172d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d6172d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52714/0 (socket says 172.21.15.49:52714) 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.077+0000 7f8d6172d640 1 -- 172.21.15.49:0/3262366314 learned_addr learned my addr 172.21.15.49:0/3262366314 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 -- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0d0180 msgr2=0x7f8d5c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0d0180 0x7f8d5c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 -- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d480039e0 msgr2=0x7f8d5c158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d480039e0 0x7f8d5c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d480039e0 0x7f8d5c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 -- 172.21.15.49:0/3262366314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d5c0d0b00 con 0x7f8d5c14a810 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d6172d640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b30d0 tx=0x7f8d4400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440053a0 con 0x7f8d5c14a810 2024-05-09T08:58:34.081 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d44018070 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d5c15df80 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44003830 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.078+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d5c0b3b40 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.079+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5c14ede0 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.080+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d440039d0 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.080+0000 7f8d597fa640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 0x7f8d3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.080+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f8d4408aa80 con 0x7f8d5c14a810 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.080+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 0x7f8d3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.082 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.081+0000 7f8d60f2c640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 0x7f8d3c06e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d54032000 tx=0x7f8d54009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.083 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.084+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d5c14ede0 con 0x7f8d5c14a810 2024-05-09T08:58:34.267 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.268+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d5c14da80 con 0x7f8d5c14a810 2024-05-09T08:58:34.268 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.269+0000 7f8d597fa640 1 -- 172.21.15.49:0/3262366314 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16838 (secure 0 0 0) 0x7f8d5c14da80 con 0x7f8d5c14a810 2024-05-09T08:58:34.268 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:34.269 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":83,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:58:30.390193+0000","last_up_change":"2024-05-09T08:58:12.270880+0000","last_in_change":"2024-05-09T08:58:13.272774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":38,"up_from":65,"up_thru":82,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6809","nonce":282207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6811","nonce":282207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6815","nonce":282207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6813","nonce":282207482}]},"public_addr":"172.21.15.49:6809/282207482","cluster_addr":"172.21.15.49:6811/282207482","heartbeat_back_addr":"172.21.15.49:6815/282207482","heartbeat_front_addr":"172.21.15.49:6813/282207482","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":39},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":6}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 msgr2=0x7f8d3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 0x7f8d3c06e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8d54032000 tx=0x7f8d54009030 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 msgr2=0x7f8d5c14ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0b30d0 tx=0x7f8d4400f920 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 shutdown_connections 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f8d3c06be60 0x7f8d3c06e320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f8d5c14a810 0x7f8d5c14ac90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d5c0d0180 0x7f8d5c0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 --2- 172.21.15.49:0/3262366314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d480039e0 0x7f8d5c158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 >> 172.21.15.49:0/3262366314 conn(0x7f8d5c06bd00 msgr2=0x7f8d5c14d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 shutdown_connections 2024-05-09T08:58:34.271 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.271+0000 7f8d63190640 1 -- 172.21.15.49:0/3262366314 wait complete. 2024-05-09T08:58:34.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T08:55:56.616182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T08:55:59.400583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T08:56:00.024628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T08:58:34.282 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T08:58:34.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.390+0000 7f7dd9cf5640 1 Processor -- start 2024-05-09T08:58:34.389 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.390+0000 7f7dd9cf5640 1 -- start start 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd9cf5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 0x7f7dd4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd9cf5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7dd40c7130 0x7f7dd40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd9cf5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd4159b10 con 0x7f7dd40c6450 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd9cf5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd4159270 con 0x7f7dd40c7130 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd9cf5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd4159550 con 0x7f7dd4152e30 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7dd40c7130 0x7f7dd40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7dd40c7130 0x7f7dd40c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52720/0 (socket says 172.21.15.49:52720) 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 -- 172.21.15.49:0/3582943896 learned_addr learned my addr 172.21.15.49:0/3582943896 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd37fe640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 0x7f7dd4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 -- 172.21.15.49:0/3582943896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 msgr2=0x7f7dd4158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 0x7f7dd4158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 -- 172.21.15.49:0/3582943896 >> v1:172.21.15.183:6789/0 conn(0x7f7dd40c6450 legacy=0x7f7dd40c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd2ffd640 1 -- 172.21.15.49:0/3582943896 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd4159830 con 0x7f7dd40c7130 2024-05-09T08:58:34.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.391+0000 7f7dd37fe640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 0x7f7dd4158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:34.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.392+0000 7f7dd2ffd640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd40c7130 0x7f7dd40c5f10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009870 tx=0x7f7dc802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed8c49ff74a7d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.392+0000 7f7dd27fc640 1 -- 172.21.15.49:0/3582943896 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8037070 con 0x7f7dd40c7130 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.392+0000 7f7dd27fc640 1 -- 172.21.15.49:0/3582943896 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc8032440 con 0x7f7dd40c7130 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.392+0000 7f7dd27fc640 1 -- 172.21.15.49:0/3582943896 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8032810 con 0x7f7dd40c7130 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/3582943896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd40c7130 msgr2=0x7f7dd40c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd40c7130 0x7f7dd40c5f10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009870 tx=0x7f7dc802a210 comp rx=0 tx=0).stop 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/3582943896 shutdown_connections 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd40c7130 0x7f7dd40c5f10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/3582943896 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd4152e30 0x7f7dd4158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/3582943896 >> 172.21.15.49:0/3582943896 conn(0x7f7dd40cc290 msgr2=0x7f7dd40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/3582943896 shutdown_connections 2024-05-09T08:58:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.393+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/3582943896 wait complete. 2024-05-09T08:58:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.394+0000 7f7dd9cf5640 1 Processor -- start 2024-05-09T08:58:34.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.394+0000 7f7dd9cf5640 1 -- start start 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd40c6450 0x7f7dd41f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dd40c7130 0x7f7dd4147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd406b4b0 con 0x7f7dd4152e30 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40c59f0 con 0x7f7dd40c7130 2024-05-09T08:58:34.397 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd9cf5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd4153210 con 0x7f7dd40c6450 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52732/0 (socket says 172.21.15.49:52732) 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 -- 172.21.15.49:0/1882363393 learned_addr learned my addr 172.21.15.49:0/1882363393 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd37fe640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd40c6450 0x7f7dd41f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd2ffd640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dd40c7130 0x7f7dd4147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 -- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd40c6450 msgr2=0x7f7dd41f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd40c6450 0x7f7dd41f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 -- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dd40c7130 msgr2=0x7f7dd4147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dd40c7130 0x7f7dd4147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.395+0000 7f7dd3fff640 1 -- 172.21.15.49:0/1882363393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd4070f60 con 0x7f7dd4152e30 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dd3fff640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00be60 tx=0x7f7dcc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc019070 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc014440 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd41490d0 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc014870 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.396+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd4151ef0 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.397+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd404f660 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.397+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7dcc01c070 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.398+0000 7f7dbbfff640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 0x7f7dc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.398+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f7dcc08af10 con 0x7f7dd4152e30 2024-05-09T08:58:34.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.398+0000 7f7dd37fe640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 0x7f7dc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.399+0000 7f7dd37fe640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 0x7f7dc006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400a990 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.402+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7dcc059810 con 0x7f7dd4152e30 2024-05-09T08:58:34.586 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.587+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7dd40c6230 con 0x7f7dd4152e30 2024-05-09T08:58:34.587 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.587+0000 7f7dbbfff640 1 -- 172.21.15.49:0/1882363393 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f7dcc059110 con 0x7f7dd4152e30 2024-05-09T08:58:34.587 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 1 2024-05-09T08:58:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 msgr2=0x7f7dc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 0x7f7dc006e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400a990 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-05-09T08:58:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 msgr2=0x7f7dd4142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.589 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00be60 tx=0x7f7dcc00bf50 comp rx=0 tx=0).stop 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 shutdown_connections 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7dc006be30 0x7f7dc006e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7dd4152e30 0x7f7dd4142f60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7dd40c7130 0x7f7dd4147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 --2- 172.21.15.49:0/1882363393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7dd40c6450 0x7f7dd41f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 >> 172.21.15.49:0/1882363393 conn(0x7f7dd40cc290 msgr2=0x7f7dd414f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 shutdown_connections 2024-05-09T08:58:34.590 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.589+0000 7f7dd9cf5640 1 -- 172.21.15.49:0/1882363393 wait complete. 2024-05-09T08:58:34.600 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T08:58:34.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.708+0000 7f137a5c8640 1 Processor -- start 2024-05-09T08:58:34.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.708+0000 7f137a5c8640 1 -- start start 2024-05-09T08:58:34.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f137a5c8640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1374155180 0x7f13740c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f137a5c8640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137415f160 con 0x7f13740c8f90 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f137a5c8640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13740c78b0 con 0x7f13740c2730 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f137a5c8640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f13740c7b90 con 0x7f1374155180 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f13737fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f13740c8f90 0x7f137406b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49670/0 (socket says 172.21.15.49:49670) 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.709+0000 7f13737fe640 1 -- 172.21.15.49:0/1277651628 learned_addr learned my addr 172.21.15.49:0/1277651628 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3932461226 0 0) 0x7f137415f160 con 0x7f13740c8f90 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 --2- 172.21.15.49:0/1277651628 >> v2:172.21.15.183:3300/0 conn(0x7f1374155180 0x7f13740c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f135c003520 con 0x7f13740c8f90 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1195159886 0 0) 0x7f135c003520 con 0x7f13740c8f90 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 >> v2:172.21.15.183:3300/0 conn(0x7f1374155180 msgr2=0x7f13740c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1277651628 >> v2:172.21.15.183:3300/0 conn(0x7f1374155180 0x7f13740c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 >> v1:172.21.15.183:6789/0 conn(0x7f13740c2730 legacy=0x7f1374165600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.712 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137415e3a0 con 0x7f13740c8f90 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f1368001670 con 0x7f13740c8f90 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 >> v1:172.21.15.49:6789/0 conn(0x7f13740c8f90 legacy=0x7f137406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f135c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f135c004800 0x7f135c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f135c0051c0 0x7f135c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13740c7b90 con 0x7f135c0039b0 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f135c00c160 con 0x7f135c0051c0 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f135c00c360 con 0x7f135c004800 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f135c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f13737fe640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f135c004800 0x7f135c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1378b65640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f135c0051c0 0x7f135c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 -- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f135c004800 msgr2=0x7f135c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.713 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f135c004800 0x7f135c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 -- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f135c0051c0 msgr2=0x7f135c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f135c0051c0 0x7f135c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1373fff640 1 -- 172.21.15.49:0/1277651628 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c003520 con 0x7f135c0039b0 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.710+0000 7f1378b65640 1 -- 172.21.15.49:0/1277651628 reap_dead start 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.711+0000 7f1373fff640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f135c003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ac0 tx=0x7f135c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81aefdb60896193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.711+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364036070 con 0x7f135c0039b0 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.711+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364031400 con 0x7f135c0039b0 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.711+0000 7f1372ffd640 1 -- 172.21.15.49:0/1277651628 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364009690 con 0x7f135c0039b0 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 -- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 msgr2=0x7f135c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f135c003dd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ac0 tx=0x7f135c005830 comp rx=0 tx=0).stop 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 -- 172.21.15.49:0/1277651628 shutdown_connections 2024-05-09T08:58:34.714 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1277651628 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f135c003dd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 -- 172.21.15.49:0/1277651628 >> 172.21.15.49:0/1277651628 conn(0x7f13740c9650 msgr2=0x7f13740c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 -- 172.21.15.49:0/1277651628 shutdown_connections 2024-05-09T08:58:34.715 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.712+0000 7f137a5c8640 1 -- 172.21.15.49:0/1277651628 wait complete. 2024-05-09T08:58:34.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.713+0000 7f137a5c8640 1 Processor -- start 2024-05-09T08:58:34.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.713+0000 7f137a5c8640 1 -- start start 2024-05-09T08:58:34.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13741f0db0 0x7f13741f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.717 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f137414e560 0x7f137414e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f13740c7550 con 0x7f135c0039b0 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f137415e960 con 0x7f13741f0db0 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f137a5c8640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f13740c32a0 con 0x7f137414e560 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f1373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f1373fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52744/0 (socket says 172.21.15.49:52744) 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f1373fff640 1 -- 172.21.15.49:0/1490072129 learned_addr learned my addr 172.21.15.49:0/1490072129 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f1373fff640 1 -- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f137414e560 msgr2=0x7f137414e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.714+0000 7f1373fff640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f137414e560 0x7f137414e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1373fff640 1 -- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13741f0db0 msgr2=0x7f13741f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1373fff640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13741f0db0 0x7f13741f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1373fff640 1 -- 172.21.15.49:0/1490072129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13741f1730 con 0x7f135c0039b0 2024-05-09T08:58:34.718 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1373fff640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1364034040 tx=0x7f1364004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136403f020 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1364037040 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13741f4350 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1364036070 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.715+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13741f4910 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.716+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137404f660 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.717+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1364042020 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.717+0000 7f1370ff9640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 0x7f136006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.717+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f13640aaf40 con 0x7f135c0039b0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.717+0000 7f13737fe640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 0x7f136006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:34.719 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.718+0000 7f13737fe640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 0x7f136006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13740c5930 tx=0x7f13680014d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:34.720 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.721+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1364049030 con 0x7f135c0039b0 2024-05-09T08:58:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.903+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13740c2200 con 0x7f135c0039b0 2024-05-09T08:58:34.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.904+0000 7f1370ff9640 1 -- 172.21.15.49:0/1490072129 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f1364079870 con 0x7f135c0039b0 2024-05-09T08:58:34.903 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 32 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 msgr2=0x7f136006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 0x7f136006e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f13740c5930 tx=0x7f13680014d0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 msgr2=0x7f137415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1364034040 tx=0x7f1364004730 comp rx=0 tx=0).stop 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 shutdown_connections 2024-05-09T08:58:34.905 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f136006be30 0x7f136006e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f137414e560 0x7f137414e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13741f0db0 0x7f13741f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 --2- 172.21.15.49:0/1490072129 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f135c0039b0 0x7f137415d310 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 >> 172.21.15.49:0/1490072129 conn(0x7f13740c9650 msgr2=0x7f1374153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 shutdown_connections 2024-05-09T08:58:34.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:34.906+0000 7f137a5c8640 1 -- 172.21.15.49:0/1490072129 wait complete. 2024-05-09T08:58:34.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T08:58:35.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.024+0000 7f657cca5640 1 Processor -- start 2024-05-09T08:58:35.023 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.024+0000 7f657cca5640 1 -- start start 2024-05-09T08:58:35.024 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657cca5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6578152ed0 0x7f6578072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657cca5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65780c1fc0 con 0x7f65780c7c60 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657cca5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65780c22a0 con 0x7f65780c3a60 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657cca5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f65780c8890 con 0x7f6578152ed0 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f6578152ed0 0x7f6578072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f6576d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f65780c7c60 0x7f6578163440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49678/0 (socket says 172.21.15.49:49678) 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f6576d9d640 1 -- 172.21.15.49:0/4147950718 learned_addr learned my addr 172.21.15.49:0/4147950718 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657659c640 1 -- 172.21.15.49:0/4147950718 >> v1:172.21.15.183:6789/0 conn(0x7f65780c3a60 legacy=0x7f65780c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657659c640 1 -- 172.21.15.49:0/4147950718 >> v1:172.21.15.49:6789/0 conn(0x7f65780c7c60 legacy=0x7f6578163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657659c640 1 -- 172.21.15.49:0/4147950718 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780c8ae0 con 0x7f6578152ed0 2024-05-09T08:58:35.025 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.025+0000 7f657559a640 1 -- 172.21.15.49:0/4147950718 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2836929114 0 0) 0x7f65780c1fc0 con 0x7f65780c7c60 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.026+0000 7f657659c640 1 --2- 172.21.15.49:0/4147950718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6578152ed0 0x7f6578072b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65680097d0 tx=0x7f656802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ede10028e9d44b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.026+0000 7f657559a640 1 -- 172.21.15.49:0/4147950718 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568037070 con 0x7f6578152ed0 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.026+0000 7f657559a640 1 -- 172.21.15.49:0/4147950718 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6568032440 con 0x7f6578152ed0 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.026+0000 7f657559a640 1 -- 172.21.15.49:0/4147950718 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568032810 con 0x7f6578152ed0 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 -- 172.21.15.49:0/4147950718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6578152ed0 msgr2=0x7f6578072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 --2- 172.21.15.49:0/4147950718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6578152ed0 0x7f6578072b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65680097d0 tx=0x7f656802a270 comp rx=0 tx=0).stop 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 -- 172.21.15.49:0/4147950718 shutdown_connections 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 --2- 172.21.15.49:0/4147950718 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6578152ed0 0x7f6578072b50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 -- 172.21.15.49:0/4147950718 >> 172.21.15.49:0/4147950718 conn(0x7f65780c9650 msgr2=0x7f65780c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 -- 172.21.15.49:0/4147950718 shutdown_connections 2024-05-09T08:58:35.027 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.027+0000 7f657cca5640 1 -- 172.21.15.49:0/4147950718 wait complete. 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.028+0000 7f657cca5640 1 Processor -- start 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.028+0000 7f657cca5640 1 -- start start 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65780c3a60 0x7f6578071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578152ed0 0x7f65780c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f65781532b0 con 0x7f65780c7c60 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65780c90a0 con 0x7f6578152ed0 2024-05-09T08:58:35.030 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657cca5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f65780c8320 con 0x7f65780c3a60 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52768/0 (socket says 172.21.15.49:52768) 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 -- 172.21.15.49:0/915704179 learned_addr learned my addr 172.21.15.49:0/915704179 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f657659c640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65780c3a60 0x7f6578071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6576d9d640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578152ed0 0x7f65780c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 -- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65780c3a60 msgr2=0x7f6578071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65780c3a60 0x7f6578071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 -- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578152ed0 msgr2=0x7f65780c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578152ed0 0x7f65780c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.029+0000 7f6575d9b640 1 -- 172.21.15.49:0/915704179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65780c3ff0 con 0x7f65780c7c60 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f6575d9b640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c450 tx=0x7f656c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0052d0 con 0x7f65780c7c60 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656c005470 con 0x7f65780c7c60 2024-05-09T08:58:35.031 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781e3a40 con 0x7f65780c7c60 2024-05-09T08:58:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c012870 con 0x7f65780c7c60 2024-05-09T08:58:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.030+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65780c4260 con 0x7f65780c7c60 2024-05-09T08:58:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.031+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657804f660 con 0x7f65780c7c60 2024-05-09T08:58:35.032 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.031+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f656c012a10 con 0x7f65780c7c60 2024-05-09T08:58:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.032+0000 7f6566ffd640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.032+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f656c08abd0 con 0x7f65780c7c60 2024-05-09T08:58:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.032+0000 7f657659c640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.033 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.032+0000 7f657659c640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 0x7f655806e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6568034000 tx=0x7f656802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.035 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.036+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f656c059440 con 0x7f65780c7c60 2024-05-09T08:58:35.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.219+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f657806ca30 con 0x7f65780c7c60 2024-05-09T08:58:35.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.219+0000 7f6566ffd640 1 -- 172.21.15.49:0/915704179 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f656c058d40 con 0x7f65780c7c60 2024-05-09T08:58:35.218 INFO:teuthology.orchestra.run.smithi049.stdout:pg_num: 64 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 msgr2=0x7f655806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 0x7f655806e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6568034000 tx=0x7f656802e000 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 msgr2=0x7f6578150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c450 tx=0x7f656c00c910 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 shutdown_connections 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f655806be30 0x7f655806e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6578152ed0 0x7f65780c4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f65780c7c60 0x7f6578150730 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 --2- 172.21.15.49:0/915704179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65780c3a60 0x7f6578071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 >> 172.21.15.49:0/915704179 conn(0x7f65780c9650 msgr2=0x7f65780c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 shutdown_connections 2024-05-09T08:58:35.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.221+0000 7f657cca5640 1 -- 172.21.15.49:0/915704179 wait complete. 2024-05-09T08:58:35.232 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T08:58:35.336 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.337+0000 7f6869bce640 1 Processor -- start 2024-05-09T08:58:35.337 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.337+0000 7f6869bce640 1 -- start start 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154b90 0x7f68640cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6864154f70 0x7f68640cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68640c6640 0x7f68640c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f68640c6fd0 con 0x7f68640c6640 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f6864072600 con 0x7f6864154f70 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6869bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f68640728e0 con 0x7f6864154b90 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6863fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68640c6640 0x7f68640c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6863fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f68640c6640 0x7f68640c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52772/0 (socket says 172.21.15.49:52772) 2024-05-09T08:58:35.338 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6863fff640 1 -- 172.21.15.49:0/2064975571 learned_addr learned my addr 172.21.15.49:0/2064975571 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f68637fe640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154b90 0x7f68640cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.338+0000 7f6862ffd640 1 --2- 172.21.15.49:0/2064975571 >> v2:172.21.15.183:3300/0 conn(0x7f6864154f70 0x7f68640cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 -- 172.21.15.49:0/2064975571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154b90 msgr2=0x7f68640cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154b90 0x7f68640cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 -- 172.21.15.49:0/2064975571 >> v2:172.21.15.183:3300/0 conn(0x7f6864154f70 msgr2=0x7f68640cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 --2- 172.21.15.49:0/2064975571 >> v2:172.21.15.183:3300/0 conn(0x7f6864154f70 0x7f68640cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 -- 172.21.15.49:0/2064975571 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68640c5880 con 0x7f68640c6640 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f6863fff640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68640c6640 0x7f68640c6a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f68640cdc30 tx=0x7f685c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bea81579224deb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f68627fc640 1 -- 172.21.15.49:0/2064975571 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c036070 con 0x7f68640c6640 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.339+0000 7f68627fc640 1 -- 172.21.15.49:0/2064975571 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c031600 con 0x7f68640c6640 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f68627fc640 1 -- 172.21.15.49:0/2064975571 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0319d0 con 0x7f68640c6640 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 -- 172.21.15.49:0/2064975571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68640c6640 msgr2=0x7f68640c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68640c6640 0x7f68640c6a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f68640cdc30 tx=0x7f685c02a730 comp rx=0 tx=0).stop 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 -- 172.21.15.49:0/2064975571 shutdown_connections 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f68640c6640 0x7f68640c6a90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 --2- 172.21.15.49:0/2064975571 >> v2:172.21.15.183:3300/0 conn(0x7f6864154f70 0x7f68640cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 --2- 172.21.15.49:0/2064975571 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154b90 0x7f68640cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 -- 172.21.15.49:0/2064975571 >> 172.21.15.49:0/2064975571 conn(0x7f6864074840 msgr2=0x7f6864074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 -- 172.21.15.49:0/2064975571 shutdown_connections 2024-05-09T08:58:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.340+0000 7f6869bce640 1 -- 172.21.15.49:0/2064975571 wait complete. 2024-05-09T08:58:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.341+0000 7f6869bce640 1 Processor -- start 2024-05-09T08:58:35.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.341+0000 7f6869bce640 1 -- start start 2024-05-09T08:58:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 0x7f6864156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154f70 0x7f686414cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f68640c6120 con 0x7f6864154b90 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68640c5e40 con 0x7f68640c6640 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6869bce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f686415af40 con 0x7f6864154f70 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6862ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6862ffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52782/0 (socket says 172.21.15.49:52782) 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6862ffd640 1 -- 172.21.15.49:0/2349936586 learned_addr learned my addr 172.21.15.49:0/2349936586 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f6863fff640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154f70 0x7f686414cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.342+0000 7f68637fe640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 0x7f6864156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 -- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154f70 msgr2=0x7f686414cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154f70 0x7f686414cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 -- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 msgr2=0x7f6864156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.345 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 0x7f6864156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 -- 172.21.15.49:0/2349936586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68641f5260 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f68637fe640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 0x7f6864156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6862ffd640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9e0 tx=0x7f685800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858017070 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685800c9d0 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685800cdc0 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68641f5540 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.343+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68641e5780 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.344+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686404f660 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.344+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f685801a070 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.345+0000 7f684bfff640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 0x7f685006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.345+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f685808b190 con 0x7f6864154b90 2024-05-09T08:58:35.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.345+0000 7f68637fe640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 0x7f685006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.345+0000 7f68637fe640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 0x7f685006e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6854009af0 tx=0x7f6854008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.349+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6858059440 con 0x7f6864154b90 2024-05-09T08:58:35.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.544+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68640cc900 con 0x7f6864154b90 2024-05-09T08:58:35.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.544+0000 7f684bfff640 1 -- 172.21.15.49:0/2349936586 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) ==== 80+0+0 (secure 0 0 0) 0x7f685805d740 con 0x7f6864154b90 2024-05-09T08:58:35.544 INFO:teuthology.orchestra.run.smithi049.stderr:listed 0 entries 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 msgr2=0x7f685006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 0x7f685006e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6854009af0 tx=0x7f6854008040 comp rx=0 tx=0).stop 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 msgr2=0x7f68641e4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f685800e9e0 tx=0x7f685800eea0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 shutdown_connections 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f685006be30 0x7f685006e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6864154f70 0x7f686414cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f6864154b90 0x7f68641e4150 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.546 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 --2- 172.21.15.49:0/2349936586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68640c6640 0x7f6864156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.546+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 >> 172.21.15.49:0/2349936586 conn(0x7f6864074840 msgr2=0x7f6864071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.547+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 shutdown_connections 2024-05-09T08:58:35.547 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.547+0000 7f6869bce640 1 -- 172.21.15.49:0/2349936586 wait complete. 2024-05-09T08:58:35.557 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T08:58:35.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:58:35.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T08:58:35.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:58:35.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:58:35.611 DEBUG:teuthology.orchestra.run.smithi049:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T08:58:35.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:58:35.646 DEBUG:teuthology.orchestra.run.smithi049:> ip netns list 2024-05-09T08:58:35.702 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T08:58:35.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:58:35.703 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T08:58:35.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T08:58:35.771 DEBUG:teuthology.orchestra.run.smithi049:> sudo ip link delete ceph-brx 2024-05-09T08:58:35.867 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T08:58:35.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T08:58:35.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:35.982 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.983+0000 7fe8cb4d4640 1 Processor -- start 2024-05-09T08:58:35.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.983+0000 7fe8cb4d4640 1 -- start start 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8cb4d4640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8c4151e00 0x7fe8c40c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8cb4d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c40c8780 0x7fe8c41622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8cb4d4640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c40c8350 con 0x7fe8c40c9c00 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8cb4d4640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c41521e0 con 0x7fe8c4151e00 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8cb4d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c4158490 con 0x7fe8c40c8780 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8c9270640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8c4151e00 0x7fe8c40c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8c9270640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe8c4151e00 0x7fe8c40c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52790/0 (socket says 172.21.15.49:52790) 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.984+0000 7fe8c9270640 1 -- 172.21.15.49:0/107658142 learned_addr learned my addr 172.21.15.49:0/107658142 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9a71640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c40c8780 0x7fe8c41622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.984 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9270640 1 -- 172.21.15.49:0/107658142 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c40c8780 msgr2=0x7fe8c41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:35.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9270640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c40c8780 0x7fe8c41622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9270640 1 -- 172.21.15.49:0/107658142 >> v1:172.21.15.49:6789/0 conn(0x7fe8c40c9c00 legacy=0x7fe8c40c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9270640 1 -- 172.21.15.49:0/107658142 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c4158750 con 0x7fe8c4151e00 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c9270640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c40c6e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009870 tx=0x7fe8bc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73687c2c53c068fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.985+0000 7fe8c3fff640 1 -- 172.21.15.49:0/107658142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc037070 con 0x7fe8c4151e00 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8c3fff640 1 -- 172.21.15.49:0/107658142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc02ae50 con 0x7fe8c4151e00 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8c3fff640 1 -- 172.21.15.49:0/107658142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc032860 con 0x7fe8c4151e00 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/107658142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 msgr2=0x7fe8c40c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c40c6e00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009870 tx=0x7fe8bc02a300 comp rx=0 tx=0).stop 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/107658142 shutdown_connections 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c40c8780 0x7fe8c41622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/107658142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c40c6e00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/107658142 >> 172.21.15.49:0/107658142 conn(0x7fe8c4074840 msgr2=0x7fe8c4074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:35.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/107658142 shutdown_connections 2024-05-09T08:58:35.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.986+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/107658142 wait complete. 2024-05-09T08:58:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.987+0000 7fe8cb4d4640 1 Processor -- start 2024-05-09T08:58:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.987+0000 7fe8cb4d4640 1 -- start start 2024-05-09T08:58:35.989 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8c40c8780 0x7fe8c40c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c4159ac0 0x7fe8c41f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4071bb0 con 0x7fe8c4151e00 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c40718d0 con 0x7fe8c40c8780 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8cb4d4640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c4071e90 con 0x7fe8c4159ac0 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8c8a6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8c8a6f640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52804/0 (socket says 172.21.15.49:52804) 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8c8a6f640 1 -- 172.21.15.49:0/3305399765 learned_addr learned my addr 172.21.15.49:0/3305399765 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8c9a71640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c4159ac0 0x7fe8c41f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.990 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.988+0000 7fe8c9270640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8c40c8780 0x7fe8c40c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 -- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c4159ac0 msgr2=0x7fe8c41f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c4159ac0 0x7fe8c41f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 -- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8c40c8780 msgr2=0x7fe8c40c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8c40c8780 0x7fe8c40c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 -- 172.21.15.49:0/3305399765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c4152910 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c8a6f640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c3c0 tx=0x7fe8b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80052d0 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8005490 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8007260 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c41f2b90 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.989+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c415b270 con 0x7fe8c4151e00 2024-05-09T08:58:35.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.990+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c40cd290 con 0x7fe8c4151e00 2024-05-09T08:58:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.991+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8b8007400 con 0x7fe8c4151e00 2024-05-09T08:58:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.991+0000 7fe8c1ffb640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 0x7fe8b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.991+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fe8b808a400 con 0x7fe8c4151e00 2024-05-09T08:58:35.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.991+0000 7fe8c9270640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 0x7fe8b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:35.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.992+0000 7fe8c9270640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 0x7fe8b006e320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc034000 tx=0x7fe8bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:35.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:35.995+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe8c40cd290 con 0x7fe8c4151e00 2024-05-09T08:58:36.174 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.175+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8c4075850 con 0x7fe8b006be60 2024-05-09T08:58:36.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.181+0000 7fe8c1ffb640 1 -- 172.21.15.49:0/3305399765 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252343 (secure 0 0 0) 0x7fe8bc07f9d0 con 0x7fe8b006be60 2024-05-09T08:58:36.183 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:36.184 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 msgr2=0x7fe8b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 0x7fe8b006e320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc034000 tx=0x7fe8bc02e000 comp rx=0 tx=0).stop 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 msgr2=0x7fe8c4159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c3c0 tx=0x7fe8b800c880 comp rx=0 tx=0).stop 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 shutdown_connections 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe8b006be60 0x7fe8b006e320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8c4159ac0 0x7fe8c41f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe8c4151e00 0x7fe8c4159580 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 --2- 172.21.15.49:0/3305399765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8c40c8780 0x7fe8c40c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 >> 172.21.15.49:0/3305399765 conn(0x7fe8c4074840 msgr2=0x7fe8c40c7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:36.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 shutdown_connections 2024-05-09T08:58:36.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.184+0000 7fe8cb4d4640 1 -- 172.21.15.49:0/3305399765 wait complete. 2024-05-09T08:58:36.194 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-05-09T08:58:34.603541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634649,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":406,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239544,"kb_used_data":9052,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735176,"statfs":{"total":773094113280,"available":772848820224,"internally_reserved":0,"allocated":9269248,"data_stored":4889600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.221564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598479+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:29.598479+0000","last_peered":"2024-05-09T08:58:29.598479+0000","last_clean":"2024-05-09T08:58:29.598479+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:29.598479+0000","last_undegraded":"2024-05-09T08:58:29.598479+0000","last_fullsized":"2024-05-09T08:58:29.598479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393481+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:30.393481+0000","last_peered":"2024-05-09T08:58:30.393481+0000","last_clean":"2024-05-09T08:58:30.393481+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:30.393481+0000","last_undegraded":"2024-05-09T08:58:30.393481+0000","last_fullsized":"2024-05-09T08:58:30.393481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395271+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:30.395271+0000","last_peered":"2024-05-09T08:58:30.395271+0000","last_clean":"2024-05-09T08:58:30.395271+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:30.395271+0000","last_undegraded":"2024-05-09T08:58:30.395271+0000","last_fullsized":"2024-05-09T08:58:30.395271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395664+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:30.395664+0000","last_peered":"2024-05-09T08:58:30.395664+0000","last_clean":"2024-05-09T08:58:30.395664+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:30.395664+0000","last_undegraded":"2024-05-09T08:58:30.395664+0000","last_fullsized":"2024-05-09T08:58:30.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395506+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395506+0000","last_peered":"2024-05-09T08:58:30.395506+0000","last_clean":"2024-05-09T08:58:30.395506+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395506+0000","last_undegraded":"2024-05-09T08:58:30.395506+0000","last_fullsized":"2024-05-09T08:58:30.395506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395310+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:30.395310+0000","last_peered":"2024-05-09T08:58:30.395310+0000","last_clean":"2024-05-09T08:58:30.395310+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:30.395310+0000","last_undegraded":"2024-05-09T08:58:30.395310+0000","last_fullsized":"2024-05-09T08:58:30.395310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396728+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396728+0000","last_peered":"2024-05-09T08:58:30.396728+0000","last_clean":"2024-05-09T08:58:30.396728+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396728+0000","last_undegraded":"2024-05-09T08:58:30.396728+0000","last_fullsized":"2024-05-09T08:58:30.396728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395275+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.395275+0000","last_peered":"2024-05-09T08:58:30.395275+0000","last_clean":"2024-05-09T08:58:30.395275+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.395275+0000","last_undegraded":"2024-05-09T08:58:30.395275+0000","last_fullsized":"2024-05-09T08:58:30.395275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394495+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:29.394495+0000","last_peered":"2024-05-09T08:58:29.394495+0000","last_clean":"2024-05-09T08:58:29.394495+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:29.394495+0000","last_undegraded":"2024-05-09T08:58:29.394495+0000","last_fullsized":"2024-05-09T08:58:29.394495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394484+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:29.394484+0000","last_peered":"2024-05-09T08:58:29.394484+0000","last_clean":"2024-05-09T08:58:29.394484+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:29.394484+0000","last_undegraded":"2024-05-09T08:58:29.394484+0000","last_fullsized":"2024-05-09T08:58:29.394484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395098+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395098+0000","last_peered":"2024-05-09T08:58:30.395098+0000","last_clean":"2024-05-09T08:58:30.395098+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395098+0000","last_undegraded":"2024-05-09T08:58:30.395098+0000","last_fullsized":"2024-05-09T08:58:30.395098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"63'3","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393249+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393249+0000","last_peered":"2024-05-09T08:58:30.393249+0000","last_clean":"2024-05-09T08:58:30.393249+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393249+0000","last_undegraded":"2024-05-09T08:58:30.393249+0000","last_fullsized":"2024-05-09T08:58:30.393249+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598225+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:29.598225+0000","last_peered":"2024-05-09T08:58:29.598225+0000","last_clean":"2024-05-09T08:58:29.598225+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:29.598225+0000","last_undegraded":"2024-05-09T08:58:29.598225+0000","last_fullsized":"2024-05-09T08:58:29.598225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396274+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396274+0000","last_peered":"2024-05-09T08:58:30.396274+0000","last_clean":"2024-05-09T08:58:30.396274+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396274+0000","last_undegraded":"2024-05-09T08:58:30.396274+0000","last_fullsized":"2024-05-09T08:58:30.396274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395329+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:30.395329+0000","last_peered":"2024-05-09T08:58:30.395329+0000","last_clean":"2024-05-09T08:58:30.395329+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:30.395329+0000","last_undegraded":"2024-05-09T08:58:30.395329+0000","last_fullsized":"2024-05-09T08:58:30.395329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394214+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:29.394214+0000","last_peered":"2024-05-09T08:58:29.394214+0000","last_clean":"2024-05-09T08:58:29.394214+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:29.394214+0000","last_undegraded":"2024-05-09T08:58:29.394214+0000","last_fullsized":"2024-05-09T08:58:29.394214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394550+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:29.394550+0000","last_peered":"2024-05-09T08:58:29.394550+0000","last_clean":"2024-05-09T08:58:29.394550+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:29.394550+0000","last_undegraded":"2024-05-09T08:58:29.394550+0000","last_fullsized":"2024-05-09T08:58:29.394550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598470+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:29.598470+0000","last_peered":"2024-05-09T08:58:29.598470+0000","last_clean":"2024-05-09T08:58:29.598470+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:29.598470+0000","last_undegraded":"2024-05-09T08:58:29.598470+0000","last_fullsized":"2024-05-09T08:58:29.598470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598367+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:29.598367+0000","last_peered":"2024-05-09T08:58:29.598367+0000","last_clean":"2024-05-09T08:58:29.598367+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:29.598367+0000","last_undegraded":"2024-05-09T08:58:29.598367+0000","last_fullsized":"2024-05-09T08:58:29.598367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393522+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:30.393522+0000","last_peered":"2024-05-09T08:58:30.393522+0000","last_clean":"2024-05-09T08:58:30.393522+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:30.393522+0000","last_undegraded":"2024-05-09T08:58:30.393522+0000","last_fullsized":"2024-05-09T08:58:30.393522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395699+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:30.395699+0000","last_peered":"2024-05-09T08:58:30.395699+0000","last_clean":"2024-05-09T08:58:30.395699+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:30.395699+0000","last_undegraded":"2024-05-09T08:58:30.395699+0000","last_fullsized":"2024-05-09T08:58:30.395699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395129+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:30.395129+0000","last_peered":"2024-05-09T08:58:30.395129+0000","last_clean":"2024-05-09T08:58:30.395129+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:30.395129+0000","last_undegraded":"2024-05-09T08:58:30.395129+0000","last_fullsized":"2024-05-09T08:58:30.395129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393980+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:29.393980+0000","last_peered":"2024-05-09T08:58:29.393980+0000","last_clean":"2024-05-09T08:58:29.393980+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:29.393980+0000","last_undegraded":"2024-05-09T08:58:29.393980+0000","last_fullsized":"2024-05-09T08:58:29.393980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394488+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:30.394488+0000","last_peered":"2024-05-09T08:58:30.394488+0000","last_clean":"2024-05-09T08:58:30.394488+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:30.394488+0000","last_undegraded":"2024-05-09T08:58:30.394488+0000","last_fullsized":"2024-05-09T08:58:30.394488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397388+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397388+0000","last_peered":"2024-05-09T08:58:30.397388+0000","last_clean":"2024-05-09T08:58:30.397388+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397388+0000","last_undegraded":"2024-05-09T08:58:30.397388+0000","last_fullsized":"2024-05-09T08:58:30.397388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393472+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393472+0000","last_peered":"2024-05-09T08:58:30.393472+0000","last_clean":"2024-05-09T08:58:30.393472+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393472+0000","last_undegraded":"2024-05-09T08:58:30.393472+0000","last_fullsized":"2024-05-09T08:58:30.393472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394161+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:29.394161+0000","last_peered":"2024-05-09T08:58:29.394161+0000","last_clean":"2024-05-09T08:58:29.394161+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:29.394161+0000","last_undegraded":"2024-05-09T08:58:29.394161+0000","last_fullsized":"2024-05-09T08:58:29.394161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396790+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:30.396790+0000","last_peered":"2024-05-09T08:58:30.396790+0000","last_clean":"2024-05-09T08:58:30.396790+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:30.396790+0000","last_undegraded":"2024-05-09T08:58:30.396790+0000","last_fullsized":"2024-05-09T08:58:30.396790+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598253+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:29.598253+0000","last_peered":"2024-05-09T08:58:29.598253+0000","last_clean":"2024-05-09T08:58:29.598253+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:29.598253+0000","last_undegraded":"2024-05-09T08:58:29.598253+0000","last_fullsized":"2024-05-09T08:58:29.598253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"63'6","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394549+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:29.394549+0000","last_peered":"2024-05-09T08:58:29.394549+0000","last_clean":"2024-05-09T08:58:29.394549+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:29.394549+0000","last_undegraded":"2024-05-09T08:58:29.394549+0000","last_fullsized":"2024-05-09T08:58:29.394549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395901+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395901+0000","last_peered":"2024-05-09T08:58:30.395901+0000","last_clean":"2024-05-09T08:58:30.395901+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395901+0000","last_undegraded":"2024-05-09T08:58:30.395901+0000","last_fullsized":"2024-05-09T08:58:30.395901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598427+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:29.598427+0000","last_peered":"2024-05-09T08:58:29.598427+0000","last_clean":"2024-05-09T08:58:29.598427+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:29.598427+0000","last_undegraded":"2024-05-09T08:58:29.598427+0000","last_fullsized":"2024-05-09T08:58:29.598427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"64'26","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394331+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:29.394331+0000","last_peered":"2024-05-09T08:58:29.394331+0000","last_clean":"2024-05-09T08:58:29.394331+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:29.394331+0000","last_undegraded":"2024-05-09T08:58:29.394331+0000","last_fullsized":"2024-05-09T08:58:29.394331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395845+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395845+0000","last_peered":"2024-05-09T08:58:30.395845+0000","last_clean":"2024-05-09T08:58:30.395845+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395845+0000","last_undegraded":"2024-05-09T08:58:30.395845+0000","last_fullsized":"2024-05-09T08:58:30.395845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395698+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395698+0000","last_peered":"2024-05-09T08:58:30.395698+0000","last_clean":"2024-05-09T08:58:30.395698+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395698+0000","last_undegraded":"2024-05-09T08:58:30.395698+0000","last_fullsized":"2024-05-09T08:58:30.395698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394169+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:29.394169+0000","last_peered":"2024-05-09T08:58:29.394169+0000","last_clean":"2024-05-09T08:58:29.394169+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:29.394169+0000","last_undegraded":"2024-05-09T08:58:29.394169+0000","last_fullsized":"2024-05-09T08:58:29.394169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394298+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:29.394298+0000","last_peered":"2024-05-09T08:58:29.394298+0000","last_clean":"2024-05-09T08:58:29.394298+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:29.394298+0000","last_undegraded":"2024-05-09T08:58:29.394298+0000","last_fullsized":"2024-05-09T08:58:29.394298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598545+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:29.598545+0000","last_peered":"2024-05-09T08:58:29.598545+0000","last_clean":"2024-05-09T08:58:29.598545+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:29.598545+0000","last_undegraded":"2024-05-09T08:58:29.598545+0000","last_fullsized":"2024-05-09T08:58:29.598545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598234+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:29.598234+0000","last_peered":"2024-05-09T08:58:29.598234+0000","last_clean":"2024-05-09T08:58:29.598234+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:29.598234+0000","last_undegraded":"2024-05-09T08:58:29.598234+0000","last_fullsized":"2024-05-09T08:58:29.598234+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394906+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:30.394906+0000","last_peered":"2024-05-09T08:58:30.394906+0000","last_clean":"2024-05-09T08:58:30.394906+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:30.394906+0000","last_undegraded":"2024-05-09T08:58:30.394906+0000","last_fullsized":"2024-05-09T08:58:30.394906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598472+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:29.598472+0000","last_peered":"2024-05-09T08:58:29.598472+0000","last_clean":"2024-05-09T08:58:29.598472+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:29.598472+0000","last_undegraded":"2024-05-09T08:58:29.598472+0000","last_fullsized":"2024-05-09T08:58:29.598472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396529+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396529+0000","last_peered":"2024-05-09T08:58:30.396529+0000","last_clean":"2024-05-09T08:58:30.396529+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396529+0000","last_undegraded":"2024-05-09T08:58:30.396529+0000","last_fullsized":"2024-05-09T08:58:30.396529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598370+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:29.598370+0000","last_peered":"2024-05-09T08:58:29.598370+0000","last_clean":"2024-05-09T08:58:29.598370+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:29.598370+0000","last_undegraded":"2024-05-09T08:58:29.598370+0000","last_fullsized":"2024-05-09T08:58:29.598370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598589+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:29.598589+0000","last_peered":"2024-05-09T08:58:29.598589+0000","last_clean":"2024-05-09T08:58:29.598589+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:29.598589+0000","last_undegraded":"2024-05-09T08:58:29.598589+0000","last_fullsized":"2024-05-09T08:58:29.598589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397204+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397204+0000","last_peered":"2024-05-09T08:58:30.397204+0000","last_clean":"2024-05-09T08:58:30.397204+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397204+0000","last_undegraded":"2024-05-09T08:58:30.397204+0000","last_fullsized":"2024-05-09T08:58:30.397204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394674+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:29.394674+0000","last_peered":"2024-05-09T08:58:29.394674+0000","last_clean":"2024-05-09T08:58:29.394674+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:29.394674+0000","last_undegraded":"2024-05-09T08:58:29.394674+0000","last_fullsized":"2024-05-09T08:58:29.394674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394637+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394637+0000","last_peered":"2024-05-09T08:58:30.394637+0000","last_clean":"2024-05-09T08:58:30.394637+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394637+0000","last_undegraded":"2024-05-09T08:58:30.394637+0000","last_fullsized":"2024-05-09T08:58:30.394637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598317+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:29.598317+0000","last_peered":"2024-05-09T08:58:29.598317+0000","last_clean":"2024-05-09T08:58:29.598317+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:29.598317+0000","last_undegraded":"2024-05-09T08:58:29.598317+0000","last_fullsized":"2024-05-09T08:58:29.598317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396900+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396900+0000","last_peered":"2024-05-09T08:58:30.396900+0000","last_clean":"2024-05-09T08:58:30.396900+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396900+0000","last_undegraded":"2024-05-09T08:58:30.396900+0000","last_fullsized":"2024-05-09T08:58:30.396900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598366+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:29.598366+0000","last_peered":"2024-05-09T08:58:29.598366+0000","last_clean":"2024-05-09T08:58:29.598366+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:29.598366+0000","last_undegraded":"2024-05-09T08:58:29.598366+0000","last_fullsized":"2024-05-09T08:58:29.598366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397263+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:30.397263+0000","last_peered":"2024-05-09T08:58:30.397263+0000","last_clean":"2024-05-09T08:58:30.397263+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:30.397263+0000","last_undegraded":"2024-05-09T08:58:30.397263+0000","last_fullsized":"2024-05-09T08:58:30.397263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393999+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:29.393999+0000","last_peered":"2024-05-09T08:58:29.393999+0000","last_clean":"2024-05-09T08:58:29.393999+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:29.393999+0000","last_undegraded":"2024-05-09T08:58:29.393999+0000","last_fullsized":"2024-05-09T08:58:29.393999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395241+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:30.395241+0000","last_peered":"2024-05-09T08:58:30.395241+0000","last_clean":"2024-05-09T08:58:30.395241+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:30.395241+0000","last_undegraded":"2024-05-09T08:58:30.395241+0000","last_fullsized":"2024-05-09T08:58:30.395241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395554+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395554+0000","last_peered":"2024-05-09T08:58:30.395554+0000","last_clean":"2024-05-09T08:58:30.395554+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395554+0000","last_undegraded":"2024-05-09T08:58:30.395554+0000","last_fullsized":"2024-05-09T08:58:30.395554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44281,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":88562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28588,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343252,"statfs":{"total":96636764160,"available":96607490048,"internally_reserved":0,"allocated":1597440,"data_stored":1044500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1028096,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":455708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":453106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":490459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874246,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29808,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342032,"statfs":{"total":96636764160,"available":96606240768,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":453308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1043018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:36.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:36.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.340+0000 7f1e4ccbd640 1 Processor -- start 2024-05-09T08:58:36.339 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.340+0000 7f1e4ccbd640 1 -- start start 2024-05-09T08:58:36.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4ccbd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e48151e00 0x7f1e480c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4ccbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e480c8780 0x7f1e481622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4ccbd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e480c8350 con 0x7f1e480c9c00 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4ccbd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f1e481521e0 con 0x7f1e48151e00 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4ccbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e48158490 con 0x7f1e480c8780 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e48151e00 0x7f1e480c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f1e48151e00 0x7f1e480c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52810/0 (socket says 172.21.15.49:52810) 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 -- 172.21.15.49:0/1181466849 learned_addr learned my addr 172.21.15.49:0/1181466849 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e46d9d640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e480c8780 0x7f1e481622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 -- 172.21.15.49:0/1181466849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e480c8780 msgr2=0x7f1e481622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e480c8780 0x7f1e481622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 -- 172.21.15.49:0/1181466849 >> v1:172.21.15.183:6789/0 conn(0x7f1e480c9c00 legacy=0x7f1e480c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.341+0000 7f1e4659c640 1 -- 172.21.15.49:0/1181466849 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e48158750 con 0x7f1e48151e00 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4659c640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e48151e00 0x7f1e480c6e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009870 tx=0x7f1e3802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32344c6ac6398eba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4559a640 1 -- 172.21.15.49:0/1181466849 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38037070 con 0x7f1e48151e00 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4559a640 1 -- 172.21.15.49:0/1181466849 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3802ae50 con 0x7f1e48151e00 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4559a640 1 -- 172.21.15.49:0/1181466849 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38032860 con 0x7f1e48151e00 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/1181466849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e48151e00 msgr2=0x7f1e480c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.343 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.342+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e48151e00 0x7f1e480c6e00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009870 tx=0x7f1e3802a300 comp rx=0 tx=0).stop 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/1181466849 shutdown_connections 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e480c8780 0x7f1e481622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/1181466849 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e48151e00 0x7f1e480c6e00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/1181466849 >> 172.21.15.49:0/1181466849 conn(0x7f1e48074840 msgr2=0x7f1e48074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/1181466849 shutdown_connections 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.343+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/1181466849 wait complete. 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.344+0000 7f1e4ccbd640 1 Processor -- start 2024-05-09T08:58:36.344 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.344+0000 7f1e4ccbd640 1 -- start start 2024-05-09T08:58:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48151e00 0x7f1e481f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.346 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 0x7f1e48141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48071bb0 con 0x7f1e480c8780 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e480718d0 con 0x7f1e48151e00 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f1e48071e90 con 0x7f1e48145b10 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52822/0 (socket says 172.21.15.49:52822) 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 -- 172.21.15.49:0/3366746999 learned_addr learned my addr 172.21.15.49:0/3366746999 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e46d9d640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 0x7f1e48141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e45d9b640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48151e00 0x7f1e481f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 -- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 msgr2=0x7f1e48141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 0x7f1e48141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 -- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48151e00 msgr2=0x7f1e481f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48151e00 0x7f1e481f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.347 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 -- 172.21.15.49:0/3366746999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4815b980 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e46d9d640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 0x7f1e48141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4659c640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1e38000c00 tx=0x7f1e380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38041070 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3803c670 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.345+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e481f3960 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.346+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38037040 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.346+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e48146350 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.346+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e48068560 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.347+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e38044070 con 0x7f1e480c8780 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.347+0000 7f1e36ffd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.348 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.347+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f1e380ab7f0 con 0x7f1e480c8780 2024-05-09T08:58:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.348+0000 7f1e45d9b640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.349 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.348+0000 7f1e45d9b640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 0x7f1e2806e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c007fd0 tx=0x7f1e3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.351 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.352+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1e3807a070 con 0x7f1e480c8780 2024-05-09T08:58:36.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.531+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e4815be60 con 0x7f1e2806be60 2024-05-09T08:58:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.538+0000 7f1e36ffd640 1 -- 172.21.15.49:0/3366746999 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252343 (secure 0 0 0) 0x7f1e3c0050d0 con 0x7f1e2806be60 2024-05-09T08:58:36.540 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:36.541 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 msgr2=0x7f1e2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 0x7f1e2806e320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c007fd0 tx=0x7f1e3c009040 comp rx=0 tx=0).stop 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 msgr2=0x7f1e481f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1e38000c00 tx=0x7f1e380046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 shutdown_connections 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f1e2806be60 0x7f1e2806e320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.543 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e48145b10 0x7f1e48141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e48151e00 0x7f1e481f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 --2- 172.21.15.49:0/3366746999 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f1e480c8780 0x7f1e481f2e00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 >> 172.21.15.49:0/3366746999 conn(0x7f1e48074840 msgr2=0x7f1e4806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 shutdown_connections 2024-05-09T08:58:36.544 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.541+0000 7f1e4ccbd640 1 -- 172.21.15.49:0/3366746999 wait complete. 2024-05-09T08:58:36.551 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-05-09T08:58:34.603541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634649,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":406,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239544,"kb_used_data":9052,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735176,"statfs":{"total":773094113280,"available":772848820224,"internally_reserved":0,"allocated":9269248,"data_stored":4889600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.221564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598479+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:29.598479+0000","last_peered":"2024-05-09T08:58:29.598479+0000","last_clean":"2024-05-09T08:58:29.598479+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:29.598479+0000","last_undegraded":"2024-05-09T08:58:29.598479+0000","last_fullsized":"2024-05-09T08:58:29.598479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393481+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:30.393481+0000","last_peered":"2024-05-09T08:58:30.393481+0000","last_clean":"2024-05-09T08:58:30.393481+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:30.393481+0000","last_undegraded":"2024-05-09T08:58:30.393481+0000","last_fullsized":"2024-05-09T08:58:30.393481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395271+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:30.395271+0000","last_peered":"2024-05-09T08:58:30.395271+0000","last_clean":"2024-05-09T08:58:30.395271+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:30.395271+0000","last_undegraded":"2024-05-09T08:58:30.395271+0000","last_fullsized":"2024-05-09T08:58:30.395271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395664+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:30.395664+0000","last_peered":"2024-05-09T08:58:30.395664+0000","last_clean":"2024-05-09T08:58:30.395664+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:30.395664+0000","last_undegraded":"2024-05-09T08:58:30.395664+0000","last_fullsized":"2024-05-09T08:58:30.395664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395506+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395506+0000","last_peered":"2024-05-09T08:58:30.395506+0000","last_clean":"2024-05-09T08:58:30.395506+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395506+0000","last_undegraded":"2024-05-09T08:58:30.395506+0000","last_fullsized":"2024-05-09T08:58:30.395506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395310+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:30.395310+0000","last_peered":"2024-05-09T08:58:30.395310+0000","last_clean":"2024-05-09T08:58:30.395310+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:30.395310+0000","last_undegraded":"2024-05-09T08:58:30.395310+0000","last_fullsized":"2024-05-09T08:58:30.395310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396728+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396728+0000","last_peered":"2024-05-09T08:58:30.396728+0000","last_clean":"2024-05-09T08:58:30.396728+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396728+0000","last_undegraded":"2024-05-09T08:58:30.396728+0000","last_fullsized":"2024-05-09T08:58:30.396728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395275+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.395275+0000","last_peered":"2024-05-09T08:58:30.395275+0000","last_clean":"2024-05-09T08:58:30.395275+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.395275+0000","last_undegraded":"2024-05-09T08:58:30.395275+0000","last_fullsized":"2024-05-09T08:58:30.395275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394495+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:29.394495+0000","last_peered":"2024-05-09T08:58:29.394495+0000","last_clean":"2024-05-09T08:58:29.394495+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:29.394495+0000","last_undegraded":"2024-05-09T08:58:29.394495+0000","last_fullsized":"2024-05-09T08:58:29.394495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394484+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:29.394484+0000","last_peered":"2024-05-09T08:58:29.394484+0000","last_clean":"2024-05-09T08:58:29.394484+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:29.394484+0000","last_undegraded":"2024-05-09T08:58:29.394484+0000","last_fullsized":"2024-05-09T08:58:29.394484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395098+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395098+0000","last_peered":"2024-05-09T08:58:30.395098+0000","last_clean":"2024-05-09T08:58:30.395098+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395098+0000","last_undegraded":"2024-05-09T08:58:30.395098+0000","last_fullsized":"2024-05-09T08:58:30.395098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"63'3","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393249+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393249+0000","last_peered":"2024-05-09T08:58:30.393249+0000","last_clean":"2024-05-09T08:58:30.393249+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393249+0000","last_undegraded":"2024-05-09T08:58:30.393249+0000","last_fullsized":"2024-05-09T08:58:30.393249+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598225+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:29.598225+0000","last_peered":"2024-05-09T08:58:29.598225+0000","last_clean":"2024-05-09T08:58:29.598225+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:29.598225+0000","last_undegraded":"2024-05-09T08:58:29.598225+0000","last_fullsized":"2024-05-09T08:58:29.598225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396274+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396274+0000","last_peered":"2024-05-09T08:58:30.396274+0000","last_clean":"2024-05-09T08:58:30.396274+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396274+0000","last_undegraded":"2024-05-09T08:58:30.396274+0000","last_fullsized":"2024-05-09T08:58:30.396274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395329+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:30.395329+0000","last_peered":"2024-05-09T08:58:30.395329+0000","last_clean":"2024-05-09T08:58:30.395329+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:30.395329+0000","last_undegraded":"2024-05-09T08:58:30.395329+0000","last_fullsized":"2024-05-09T08:58:30.395329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394214+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:29.394214+0000","last_peered":"2024-05-09T08:58:29.394214+0000","last_clean":"2024-05-09T08:58:29.394214+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:29.394214+0000","last_undegraded":"2024-05-09T08:58:29.394214+0000","last_fullsized":"2024-05-09T08:58:29.394214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394550+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:29.394550+0000","last_peered":"2024-05-09T08:58:29.394550+0000","last_clean":"2024-05-09T08:58:29.394550+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:29.394550+0000","last_undegraded":"2024-05-09T08:58:29.394550+0000","last_fullsized":"2024-05-09T08:58:29.394550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598470+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:29.598470+0000","last_peered":"2024-05-09T08:58:29.598470+0000","last_clean":"2024-05-09T08:58:29.598470+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:29.598470+0000","last_undegraded":"2024-05-09T08:58:29.598470+0000","last_fullsized":"2024-05-09T08:58:29.598470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":122,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598367+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:29.598367+0000","last_peered":"2024-05-09T08:58:29.598367+0000","last_clean":"2024-05-09T08:58:29.598367+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:29.598367+0000","last_undegraded":"2024-05-09T08:58:29.598367+0000","last_fullsized":"2024-05-09T08:58:29.598367+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393522+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:30.393522+0000","last_peered":"2024-05-09T08:58:30.393522+0000","last_clean":"2024-05-09T08:58:30.393522+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:30.393522+0000","last_undegraded":"2024-05-09T08:58:30.393522+0000","last_fullsized":"2024-05-09T08:58:30.393522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395699+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:30.395699+0000","last_peered":"2024-05-09T08:58:30.395699+0000","last_clean":"2024-05-09T08:58:30.395699+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:30.395699+0000","last_undegraded":"2024-05-09T08:58:30.395699+0000","last_fullsized":"2024-05-09T08:58:30.395699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":53,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395129+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:30.395129+0000","last_peered":"2024-05-09T08:58:30.395129+0000","last_clean":"2024-05-09T08:58:30.395129+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:30.395129+0000","last_undegraded":"2024-05-09T08:58:30.395129+0000","last_fullsized":"2024-05-09T08:58:30.395129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393980+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:29.393980+0000","last_peered":"2024-05-09T08:58:29.393980+0000","last_clean":"2024-05-09T08:58:29.393980+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:29.393980+0000","last_undegraded":"2024-05-09T08:58:29.393980+0000","last_fullsized":"2024-05-09T08:58:29.393980+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394488+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:30.394488+0000","last_peered":"2024-05-09T08:58:30.394488+0000","last_clean":"2024-05-09T08:58:30.394488+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:30.394488+0000","last_undegraded":"2024-05-09T08:58:30.394488+0000","last_fullsized":"2024-05-09T08:58:30.394488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397388+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397388+0000","last_peered":"2024-05-09T08:58:30.397388+0000","last_clean":"2024-05-09T08:58:30.397388+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397388+0000","last_undegraded":"2024-05-09T08:58:30.397388+0000","last_fullsized":"2024-05-09T08:58:30.397388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393472+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393472+0000","last_peered":"2024-05-09T08:58:30.393472+0000","last_clean":"2024-05-09T08:58:30.393472+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393472+0000","last_undegraded":"2024-05-09T08:58:30.393472+0000","last_fullsized":"2024-05-09T08:58:30.393472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394161+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:29.394161+0000","last_peered":"2024-05-09T08:58:29.394161+0000","last_clean":"2024-05-09T08:58:29.394161+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:29.394161+0000","last_undegraded":"2024-05-09T08:58:29.394161+0000","last_fullsized":"2024-05-09T08:58:29.394161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396790+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:30.396790+0000","last_peered":"2024-05-09T08:58:30.396790+0000","last_clean":"2024-05-09T08:58:30.396790+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:30.396790+0000","last_undegraded":"2024-05-09T08:58:30.396790+0000","last_fullsized":"2024-05-09T08:58:30.396790+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598253+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:29.598253+0000","last_peered":"2024-05-09T08:58:29.598253+0000","last_clean":"2024-05-09T08:58:29.598253+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:29.598253+0000","last_undegraded":"2024-05-09T08:58:29.598253+0000","last_fullsized":"2024-05-09T08:58:29.598253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"63'6","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394549+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:29.394549+0000","last_peered":"2024-05-09T08:58:29.394549+0000","last_clean":"2024-05-09T08:58:29.394549+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:29.394549+0000","last_undegraded":"2024-05-09T08:58:29.394549+0000","last_fullsized":"2024-05-09T08:58:29.394549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395901+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395901+0000","last_peered":"2024-05-09T08:58:30.395901+0000","last_clean":"2024-05-09T08:58:30.395901+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395901+0000","last_undegraded":"2024-05-09T08:58:30.395901+0000","last_fullsized":"2024-05-09T08:58:30.395901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598427+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:29.598427+0000","last_peered":"2024-05-09T08:58:29.598427+0000","last_clean":"2024-05-09T08:58:29.598427+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:29.598427+0000","last_undegraded":"2024-05-09T08:58:29.598427+0000","last_fullsized":"2024-05-09T08:58:29.598427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"64'26","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394331+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:29.394331+0000","last_peered":"2024-05-09T08:58:29.394331+0000","last_clean":"2024-05-09T08:58:29.394331+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:29.394331+0000","last_undegraded":"2024-05-09T08:58:29.394331+0000","last_fullsized":"2024-05-09T08:58:29.394331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37300,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395845+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395845+0000","last_peered":"2024-05-09T08:58:30.395845+0000","last_clean":"2024-05-09T08:58:30.395845+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395845+0000","last_undegraded":"2024-05-09T08:58:30.395845+0000","last_fullsized":"2024-05-09T08:58:30.395845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395698+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395698+0000","last_peered":"2024-05-09T08:58:30.395698+0000","last_clean":"2024-05-09T08:58:30.395698+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395698+0000","last_undegraded":"2024-05-09T08:58:30.395698+0000","last_fullsized":"2024-05-09T08:58:30.395698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394169+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:29.394169+0000","last_peered":"2024-05-09T08:58:29.394169+0000","last_clean":"2024-05-09T08:58:29.394169+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:29.394169+0000","last_undegraded":"2024-05-09T08:58:29.394169+0000","last_fullsized":"2024-05-09T08:58:29.394169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394298+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:29.394298+0000","last_peered":"2024-05-09T08:58:29.394298+0000","last_clean":"2024-05-09T08:58:29.394298+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:29.394298+0000","last_undegraded":"2024-05-09T08:58:29.394298+0000","last_fullsized":"2024-05-09T08:58:29.394298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":140,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598545+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:29.598545+0000","last_peered":"2024-05-09T08:58:29.598545+0000","last_clean":"2024-05-09T08:58:29.598545+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:29.598545+0000","last_undegraded":"2024-05-09T08:58:29.598545+0000","last_fullsized":"2024-05-09T08:58:29.598545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598234+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:29.598234+0000","last_peered":"2024-05-09T08:58:29.598234+0000","last_clean":"2024-05-09T08:58:29.598234+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:29.598234+0000","last_undegraded":"2024-05-09T08:58:29.598234+0000","last_fullsized":"2024-05-09T08:58:29.598234+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394906+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:30.394906+0000","last_peered":"2024-05-09T08:58:30.394906+0000","last_clean":"2024-05-09T08:58:30.394906+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:30.394906+0000","last_undegraded":"2024-05-09T08:58:30.394906+0000","last_fullsized":"2024-05-09T08:58:30.394906+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598472+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:29.598472+0000","last_peered":"2024-05-09T08:58:29.598472+0000","last_clean":"2024-05-09T08:58:29.598472+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:29.598472+0000","last_undegraded":"2024-05-09T08:58:29.598472+0000","last_fullsized":"2024-05-09T08:58:29.598472+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396529+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396529+0000","last_peered":"2024-05-09T08:58:30.396529+0000","last_clean":"2024-05-09T08:58:30.396529+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396529+0000","last_undegraded":"2024-05-09T08:58:30.396529+0000","last_fullsized":"2024-05-09T08:58:30.396529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":142,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598370+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:29.598370+0000","last_peered":"2024-05-09T08:58:29.598370+0000","last_clean":"2024-05-09T08:58:29.598370+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:29.598370+0000","last_undegraded":"2024-05-09T08:58:29.598370+0000","last_fullsized":"2024-05-09T08:58:29.598370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598589+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:29.598589+0000","last_peered":"2024-05-09T08:58:29.598589+0000","last_clean":"2024-05-09T08:58:29.598589+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:29.598589+0000","last_undegraded":"2024-05-09T08:58:29.598589+0000","last_fullsized":"2024-05-09T08:58:29.598589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397204+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397204+0000","last_peered":"2024-05-09T08:58:30.397204+0000","last_clean":"2024-05-09T08:58:30.397204+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397204+0000","last_undegraded":"2024-05-09T08:58:30.397204+0000","last_fullsized":"2024-05-09T08:58:30.397204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.394674+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:29.394674+0000","last_peered":"2024-05-09T08:58:29.394674+0000","last_clean":"2024-05-09T08:58:29.394674+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:29.394674+0000","last_undegraded":"2024-05-09T08:58:29.394674+0000","last_fullsized":"2024-05-09T08:58:29.394674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394637+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394637+0000","last_peered":"2024-05-09T08:58:30.394637+0000","last_clean":"2024-05-09T08:58:30.394637+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394637+0000","last_undegraded":"2024-05-09T08:58:30.394637+0000","last_fullsized":"2024-05-09T08:58:30.394637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":144,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598317+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:29.598317+0000","last_peered":"2024-05-09T08:58:29.598317+0000","last_clean":"2024-05-09T08:58:29.598317+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:29.598317+0000","last_undegraded":"2024-05-09T08:58:29.598317+0000","last_fullsized":"2024-05-09T08:58:29.598317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396900+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396900+0000","last_peered":"2024-05-09T08:58:30.396900+0000","last_clean":"2024-05-09T08:58:30.396900+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396900+0000","last_undegraded":"2024-05-09T08:58:30.396900+0000","last_fullsized":"2024-05-09T08:58:30.396900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":141,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.598366+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:29.598366+0000","last_peered":"2024-05-09T08:58:29.598366+0000","last_clean":"2024-05-09T08:58:29.598366+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:29.598366+0000","last_undegraded":"2024-05-09T08:58:29.598366+0000","last_fullsized":"2024-05-09T08:58:29.598366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397263+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:30.397263+0000","last_peered":"2024-05-09T08:58:30.397263+0000","last_clean":"2024-05-09T08:58:30.397263+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:30.397263+0000","last_undegraded":"2024-05-09T08:58:30.397263+0000","last_fullsized":"2024-05-09T08:58:30.397263+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":133,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T08:58:29.393999+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:29.393999+0000","last_peered":"2024-05-09T08:58:29.393999+0000","last_clean":"2024-05-09T08:58:29.393999+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:29.393999+0000","last_undegraded":"2024-05-09T08:58:29.393999+0000","last_fullsized":"2024-05-09T08:58:29.393999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395241+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:30.395241+0000","last_peered":"2024-05-09T08:58:30.395241+0000","last_clean":"2024-05-09T08:58:30.395241+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:30.395241+0000","last_undegraded":"2024-05-09T08:58:30.395241+0000","last_fullsized":"2024-05-09T08:58:30.395241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395554+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395554+0000","last_peered":"2024-05-09T08:58:30.395554+0000","last_clean":"2024-05-09T08:58:30.395554+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395554+0000","last_undegraded":"2024-05-09T08:58:30.395554+0000","last_fullsized":"2024-05-09T08:58:30.395554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44281,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":245,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":88562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28588,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343252,"statfs":{"total":96636764160,"available":96607490048,"internally_reserved":0,"allocated":1597440,"data_stored":1044500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1028096,"data_stored":489866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":455708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":453106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1036288,"data_stored":490459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874246,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29808,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342032,"statfs":{"total":96636764160,"available":96606240768,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":453308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1043018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:36.555 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T08:58:36.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.696+0000 7f22e9521640 1 Processor -- start 2024-05-09T08:58:36.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.696+0000 7f22e9521640 1 -- start start 2024-05-09T08:58:36.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e9521640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f22e41517f0 0x7f22e415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e9521640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22e40cc190 0x7f22e40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e9521640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22e40ccb20 con 0x7f22e4075200 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e9521640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f22e4151bd0 con 0x7f22e40cc190 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e9521640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f22e415e260 con 0x7f22e41517f0 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22e40cc190 0x7f22e40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f22e40cc190 0x7f22e40cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52838/0 (socket says 172.21.15.49:52838) 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 -- 172.21.15.49:0/1585978536 learned_addr learned my addr 172.21.15.49:0/1585978536 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 -- 172.21.15.49:0/1585978536 >> v2:172.21.15.183:3300/0 conn(0x7f22e41517f0 msgr2=0x7f22e415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 --2- 172.21.15.49:0/1585978536 >> v2:172.21.15.183:3300/0 conn(0x7f22e41517f0 0x7f22e415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 -- 172.21.15.49:0/1585978536 >> v1:172.21.15.49:6789/0 conn(0x7f22e4075200 legacy=0x7f22e40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.697+0000 7f22e37fe640 1 -- 172.21.15.49:0/1585978536 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e415e520 con 0x7f22e40cc190 2024-05-09T08:58:36.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e37fe640 1 --2- 172.21.15.49:0/1585978536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e40cc5e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22e40c78a0 tx=0x7f22d402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5534a89293b017f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e1ffb640 1 -- 172.21.15.49:0/1585978536 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4037070 con 0x7f22e40cc190 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e1ffb640 1 -- 172.21.15.49:0/1585978536 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d402ae50 con 0x7f22e40cc190 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e1ffb640 1 -- 172.21.15.49:0/1585978536 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4032860 con 0x7f22e40cc190 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e9521640 1 -- 172.21.15.49:0/1585978536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 msgr2=0x7f22e40cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.698+0000 7f22e9521640 1 --2- 172.21.15.49:0/1585978536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e40cc5e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22e40c78a0 tx=0x7f22d402a300 comp rx=0 tx=0).stop 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 -- 172.21.15.49:0/1585978536 shutdown_connections 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 --2- 172.21.15.49:0/1585978536 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e40cc5e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 --2- 172.21.15.49:0/1585978536 >> v2:172.21.15.183:3300/0 conn(0x7f22e41517f0 0x7f22e415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 -- 172.21.15.49:0/1585978536 >> 172.21.15.49:0/1585978536 conn(0x7f22e406bd00 msgr2=0x7f22e4074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 -- 172.21.15.49:0/1585978536 shutdown_connections 2024-05-09T08:58:36.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.699+0000 7f22e9521640 1 -- 172.21.15.49:0/1585978536 wait complete. 2024-05-09T08:58:36.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.700+0000 7f22e9521640 1 Processor -- start 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.700+0000 7f22e9521640 1 -- start start 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 0x7f22e41f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22e41517f0 0x7f22e4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f22e415edc0 con 0x7f22e40cc190 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22e415eae0 con 0x7f22e41517f0 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f22e415f0a0 con 0x7f22e4075200 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52844/0 (socket says 172.21.15.49:52844) 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 -- 172.21.15.49:0/857812029 learned_addr learned my addr 172.21.15.49:0/857812029 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e2ffd640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 0x7f22e41f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e37fe640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22e41517f0 0x7f22e4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 -- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 msgr2=0x7f22e41f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 0x7f22e41f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 -- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22e41517f0 msgr2=0x7f22e4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22e41517f0 0x7f22e4143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 -- 172.21.15.49:0/857812029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e41e45b0 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e2ffd640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 0x7f22e41f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e27fc640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c400 tx=0x7f22cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0052d0 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc005490 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc007260 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.701+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e41e4870 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.702+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41f5000 con 0x7f22e40cc190 2024-05-09T08:58:36.703 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.702+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e4068560 con 0x7f22e40cc190 2024-05-09T08:58:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.703+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22cc007400 con 0x7f22e40cc190 2024-05-09T08:58:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.703+0000 7f22db7fe640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 0x7f22c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:36.704 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.704+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f22cc08a3c0 con 0x7f22e40cc190 2024-05-09T08:58:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.704+0000 7f22e2ffd640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 0x7f22c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:36.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.705+0000 7f22e2ffd640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 0x7f22c406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fa0 tx=0x7f22dc009570 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:36.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.708+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22cc058cc0 con 0x7f22e40cc190 2024-05-09T08:58:36.887 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.888+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22e41e4b50 con 0x7f22c406be30 2024-05-09T08:58:36.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.895+0000 7f22db7fe640 1 -- 172.21.15.49:0/857812029 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252461 (secure 0 0 0) 0x7f22e41e4b50 con 0x7f22c406be30 2024-05-09T08:58:36.897 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:36.898 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.897+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 msgr2=0x7f22c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 0x7f22c406e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fa0 tx=0x7f22dc009570 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 msgr2=0x7f22e414b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c400 tx=0x7f22cc00c8c0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 shutdown_connections 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f22c406be30 0x7f22c406e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22e41517f0 0x7f22e4143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f22e40cc190 0x7f22e414b910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 --2- 172.21.15.49:0/857812029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22e4075200 0x7f22e41f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 >> 172.21.15.49:0/857812029 conn(0x7f22e406bd00 msgr2=0x7f22e41505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 shutdown_connections 2024-05-09T08:58:36.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:36.898+0000 7f22e9521640 1 -- 172.21.15.49:0/857812029 wait complete. 2024-05-09T08:58:36.908 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-05-09T08:58:36.603972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634963,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":410,"num_write_kb":2540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":498,"ondisk_log_size":498,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239564,"kb_used_data":9136,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735156,"statfs":{"total":773094113280,"available":772848799744,"internally_reserved":0,"allocated":9355264,"data_stored":4932258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001787"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501060+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:32.501060+0000","last_peered":"2024-05-09T08:58:32.501060+0000","last_clean":"2024-05-09T08:58:32.501060+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:32.501060+0000","last_undegraded":"2024-05-09T08:58:32.501060+0000","last_fullsized":"2024-05-09T08:58:32.501060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"83'3","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606913+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:35.606913+0000","last_peered":"2024-05-09T08:58:35.606913+0000","last_clean":"2024-05-09T08:58:35.606913+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:35.606913+0000","last_undegraded":"2024-05-09T08:58:35.606913+0000","last_fullsized":"2024-05-09T08:58:35.606913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.629232+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:35.629232+0000","last_peered":"2024-05-09T08:58:35.629232+0000","last_clean":"2024-05-09T08:58:35.629232+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:35.629232+0000","last_undegraded":"2024-05-09T08:58:35.629232+0000","last_fullsized":"2024-05-09T08:58:35.629232+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"83'1","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.601095+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:35.601095+0000","last_peered":"2024-05-09T08:58:35.601095+0000","last_clean":"2024-05-09T08:58:35.601095+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:35.601095+0000","last_undegraded":"2024-05-09T08:58:35.601095+0000","last_fullsized":"2024-05-09T08:58:35.601095+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'1","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395562+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395562+0000","last_peered":"2024-05-09T08:58:30.395562+0000","last_clean":"2024-05-09T08:58:30.395562+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395562+0000","last_undegraded":"2024-05-09T08:58:30.395562+0000","last_fullsized":"2024-05-09T08:58:30.395562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.621069+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:35.621069+0000","last_peered":"2024-05-09T08:58:35.621069+0000","last_clean":"2024-05-09T08:58:35.621069+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:35.621069+0000","last_undegraded":"2024-05-09T08:58:35.621069+0000","last_fullsized":"2024-05-09T08:58:35.621069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"83'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396779+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396779+0000","last_peered":"2024-05-09T08:58:30.396779+0000","last_clean":"2024-05-09T08:58:30.396779+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396779+0000","last_undegraded":"2024-05-09T08:58:30.396779+0000","last_fullsized":"2024-05-09T08:58:30.396779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.606929+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.606929+0000","last_peered":"2024-05-09T08:58:30.606929+0000","last_clean":"2024-05-09T08:58:30.606929+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.606929+0000","last_undegraded":"2024-05-09T08:58:30.606929+0000","last_fullsized":"2024-05-09T08:58:30.606929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217571+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:33.217571+0000","last_peered":"2024-05-09T08:58:33.217571+0000","last_clean":"2024-05-09T08:58:33.217571+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:33.217571+0000","last_undegraded":"2024-05-09T08:58:33.217571+0000","last_fullsized":"2024-05-09T08:58:33.217571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217478+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:33.217478+0000","last_peered":"2024-05-09T08:58:33.217478+0000","last_clean":"2024-05-09T08:58:33.217478+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:33.217478+0000","last_undegraded":"2024-05-09T08:58:33.217478+0000","last_fullsized":"2024-05-09T08:58:33.217478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'4","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395143+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395143+0000","last_peered":"2024-05-09T08:58:30.395143+0000","last_clean":"2024-05-09T08:58:30.395143+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395143+0000","last_undegraded":"2024-05-09T08:58:30.395143+0000","last_fullsized":"2024-05-09T08:58:30.395143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393293+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393293+0000","last_peered":"2024-05-09T08:58:30.393293+0000","last_clean":"2024-05-09T08:58:30.393293+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393293+0000","last_undegraded":"2024-05-09T08:58:30.393293+0000","last_fullsized":"2024-05-09T08:58:30.393293+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'3","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500833+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:32.500833+0000","last_peered":"2024-05-09T08:58:32.500833+0000","last_clean":"2024-05-09T08:58:32.500833+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:32.500833+0000","last_undegraded":"2024-05-09T08:58:32.500833+0000","last_fullsized":"2024-05-09T08:58:32.500833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396315+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396315+0000","last_peered":"2024-05-09T08:58:30.396315+0000","last_clean":"2024-05-09T08:58:30.396315+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396315+0000","last_undegraded":"2024-05-09T08:58:30.396315+0000","last_fullsized":"2024-05-09T08:58:30.396315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.623621+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:35.623621+0000","last_peered":"2024-05-09T08:58:35.623621+0000","last_clean":"2024-05-09T08:58:35.623621+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:35.623621+0000","last_undegraded":"2024-05-09T08:58:35.623621+0000","last_fullsized":"2024-05-09T08:58:35.623621+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.603643+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:35.603643+0000","last_peered":"2024-05-09T08:58:35.603643+0000","last_clean":"2024-05-09T08:58:35.603643+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:35.603643+0000","last_undegraded":"2024-05-09T08:58:35.603643+0000","last_fullsized":"2024-05-09T08:58:35.603643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217614+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:33.217614+0000","last_peered":"2024-05-09T08:58:33.217614+0000","last_clean":"2024-05-09T08:58:33.217614+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:33.217614+0000","last_undegraded":"2024-05-09T08:58:33.217614+0000","last_fullsized":"2024-05-09T08:58:33.217614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'3","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605574+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:35.605574+0000","last_peered":"2024-05-09T08:58:35.605574+0000","last_clean":"2024-05-09T08:58:35.605574+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:35.605574+0000","last_undegraded":"2024-05-09T08:58:35.605574+0000","last_fullsized":"2024-05-09T08:58:35.605574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501562+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:32.501562+0000","last_peered":"2024-05-09T08:58:32.501562+0000","last_clean":"2024-05-09T08:58:32.501562+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:32.501562+0000","last_undegraded":"2024-05-09T08:58:32.501562+0000","last_fullsized":"2024-05-09T08:58:32.501562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"83'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.604133+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:35.604133+0000","last_peered":"2024-05-09T08:58:35.604133+0000","last_clean":"2024-05-09T08:58:35.604133+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:35.604133+0000","last_undegraded":"2024-05-09T08:58:35.604133+0000","last_fullsized":"2024-05-09T08:58:35.604133+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'4","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.603005+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:35.603005+0000","last_peered":"2024-05-09T08:58:35.603005+0000","last_clean":"2024-05-09T08:58:35.603005+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:35.603005+0000","last_undegraded":"2024-05-09T08:58:35.603005+0000","last_fullsized":"2024-05-09T08:58:35.603005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.633084+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:35.633084+0000","last_peered":"2024-05-09T08:58:35.633084+0000","last_clean":"2024-05-09T08:58:35.633084+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:35.633084+0000","last_undegraded":"2024-05-09T08:58:35.633084+0000","last_fullsized":"2024-05-09T08:58:35.633084+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606282+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:35.606282+0000","last_peered":"2024-05-09T08:58:35.606282+0000","last_clean":"2024-05-09T08:58:35.606282+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:35.606282+0000","last_undegraded":"2024-05-09T08:58:35.606282+0000","last_fullsized":"2024-05-09T08:58:35.606282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606072+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:35.606072+0000","last_peered":"2024-05-09T08:58:35.606072+0000","last_clean":"2024-05-09T08:58:35.606072+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:35.606072+0000","last_undegraded":"2024-05-09T08:58:35.606072+0000","last_fullsized":"2024-05-09T08:58:35.606072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397438+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397438+0000","last_peered":"2024-05-09T08:58:30.397438+0000","last_clean":"2024-05-09T08:58:30.397438+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397438+0000","last_undegraded":"2024-05-09T08:58:30.397438+0000","last_fullsized":"2024-05-09T08:58:30.397438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"83'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393513+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393513+0000","last_peered":"2024-05-09T08:58:30.393513+0000","last_clean":"2024-05-09T08:58:30.393513+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393513+0000","last_undegraded":"2024-05-09T08:58:30.393513+0000","last_fullsized":"2024-05-09T08:58:30.393513+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216857+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:33.216857+0000","last_peered":"2024-05-09T08:58:33.216857+0000","last_clean":"2024-05-09T08:58:33.216857+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:33.216857+0000","last_undegraded":"2024-05-09T08:58:33.216857+0000","last_fullsized":"2024-05-09T08:58:33.216857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"83'4","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605756+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:35.605756+0000","last_peered":"2024-05-09T08:58:35.605756+0000","last_clean":"2024-05-09T08:58:35.605756+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:35.605756+0000","last_undegraded":"2024-05-09T08:58:35.605756+0000","last_fullsized":"2024-05-09T08:58:35.605756+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500695+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:32.500695+0000","last_peered":"2024-05-09T08:58:32.500695+0000","last_clean":"2024-05-09T08:58:32.500695+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:32.500695+0000","last_undegraded":"2024-05-09T08:58:32.500695+0000","last_fullsized":"2024-05-09T08:58:32.500695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217039+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:33.217039+0000","last_peered":"2024-05-09T08:58:33.217039+0000","last_clean":"2024-05-09T08:58:33.217039+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:33.217039+0000","last_undegraded":"2024-05-09T08:58:33.217039+0000","last_fullsized":"2024-05-09T08:58:33.217039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395944+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395944+0000","last_peered":"2024-05-09T08:58:30.395944+0000","last_clean":"2024-05-09T08:58:30.395944+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395944+0000","last_undegraded":"2024-05-09T08:58:30.395944+0000","last_fullsized":"2024-05-09T08:58:30.395944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501356+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:32.501356+0000","last_peered":"2024-05-09T08:58:32.501356+0000","last_clean":"2024-05-09T08:58:32.501356+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:32.501356+0000","last_undegraded":"2024-05-09T08:58:32.501356+0000","last_fullsized":"2024-05-09T08:58:32.501356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"83'8","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.607701+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:35.607701+0000","last_peered":"2024-05-09T08:58:35.607701+0000","last_clean":"2024-05-09T08:58:35.607701+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:35.607701+0000","last_undegraded":"2024-05-09T08:58:35.607701+0000","last_fullsized":"2024-05-09T08:58:35.607701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":165,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.589905+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:35.589905+0000","last_peered":"2024-05-09T08:58:35.589905+0000","last_clean":"2024-05-09T08:58:35.589905+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:35.589905+0000","last_undegraded":"2024-05-09T08:58:35.589905+0000","last_fullsized":"2024-05-09T08:58:35.589905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37473,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395893+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395893+0000","last_peered":"2024-05-09T08:58:30.395893+0000","last_clean":"2024-05-09T08:58:30.395893+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395893+0000","last_undegraded":"2024-05-09T08:58:30.395893+0000","last_fullsized":"2024-05-09T08:58:30.395893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395739+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395739+0000","last_peered":"2024-05-09T08:58:30.395739+0000","last_clean":"2024-05-09T08:58:30.395739+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395739+0000","last_undegraded":"2024-05-09T08:58:30.395739+0000","last_fullsized":"2024-05-09T08:58:30.395739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216858+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:33.216858+0000","last_peered":"2024-05-09T08:58:33.216858+0000","last_clean":"2024-05-09T08:58:33.216858+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:33.216858+0000","last_undegraded":"2024-05-09T08:58:33.216858+0000","last_fullsized":"2024-05-09T08:58:33.216858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606462+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:35.606462+0000","last_peered":"2024-05-09T08:58:35.606462+0000","last_clean":"2024-05-09T08:58:35.606462+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:35.606462+0000","last_undegraded":"2024-05-09T08:58:35.606462+0000","last_fullsized":"2024-05-09T08:58:35.606462+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501455+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:32.501455+0000","last_peered":"2024-05-09T08:58:32.501455+0000","last_clean":"2024-05-09T08:58:32.501455+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:32.501455+0000","last_undegraded":"2024-05-09T08:58:32.501455+0000","last_fullsized":"2024-05-09T08:58:32.501455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"83'3","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.502038+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:32.502038+0000","last_peered":"2024-05-09T08:58:32.502038+0000","last_clean":"2024-05-09T08:58:32.502038+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:32.502038+0000","last_undegraded":"2024-05-09T08:58:32.502038+0000","last_fullsized":"2024-05-09T08:58:32.502038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605857+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:35.605857+0000","last_peered":"2024-05-09T08:58:35.605857+0000","last_clean":"2024-05-09T08:58:35.605857+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:35.605857+0000","last_undegraded":"2024-05-09T08:58:35.605857+0000","last_fullsized":"2024-05-09T08:58:35.605857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501749+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:32.501749+0000","last_peered":"2024-05-09T08:58:32.501749+0000","last_clean":"2024-05-09T08:58:32.501749+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:32.501749+0000","last_undegraded":"2024-05-09T08:58:32.501749+0000","last_fullsized":"2024-05-09T08:58:32.501749+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396574+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396574+0000","last_peered":"2024-05-09T08:58:30.396574+0000","last_clean":"2024-05-09T08:58:30.396574+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396574+0000","last_undegraded":"2024-05-09T08:58:30.396574+0000","last_fullsized":"2024-05-09T08:58:30.396574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501652+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:32.501652+0000","last_peered":"2024-05-09T08:58:32.501652+0000","last_clean":"2024-05-09T08:58:32.501652+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:32.501652+0000","last_undegraded":"2024-05-09T08:58:32.501652+0000","last_fullsized":"2024-05-09T08:58:32.501652+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501154+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:32.501154+0000","last_peered":"2024-05-09T08:58:32.501154+0000","last_clean":"2024-05-09T08:58:32.501154+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:32.501154+0000","last_undegraded":"2024-05-09T08:58:32.501154+0000","last_fullsized":"2024-05-09T08:58:32.501154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397253+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397253+0000","last_peered":"2024-05-09T08:58:30.397253+0000","last_clean":"2024-05-09T08:58:30.397253+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397253+0000","last_undegraded":"2024-05-09T08:58:30.397253+0000","last_fullsized":"2024-05-09T08:58:30.397253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216985+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:33.216985+0000","last_peered":"2024-05-09T08:58:33.216985+0000","last_clean":"2024-05-09T08:58:33.216985+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:33.216985+0000","last_undegraded":"2024-05-09T08:58:33.216985+0000","last_fullsized":"2024-05-09T08:58:33.216985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"83'1","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394689+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394689+0000","last_peered":"2024-05-09T08:58:30.394689+0000","last_clean":"2024-05-09T08:58:30.394689+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394689+0000","last_undegraded":"2024-05-09T08:58:30.394689+0000","last_fullsized":"2024-05-09T08:58:30.394689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500883+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:32.500883+0000","last_peered":"2024-05-09T08:58:32.500883+0000","last_clean":"2024-05-09T08:58:32.500883+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:32.500883+0000","last_undegraded":"2024-05-09T08:58:32.500883+0000","last_fullsized":"2024-05-09T08:58:32.500883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"83'6","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396942+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396942+0000","last_peered":"2024-05-09T08:58:30.396942+0000","last_clean":"2024-05-09T08:58:30.396942+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396942+0000","last_undegraded":"2024-05-09T08:58:30.396942+0000","last_fullsized":"2024-05-09T08:58:30.396942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500573+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:32.500573+0000","last_peered":"2024-05-09T08:58:32.500573+0000","last_clean":"2024-05-09T08:58:32.500573+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:32.500573+0000","last_undegraded":"2024-05-09T08:58:32.500573+0000","last_fullsized":"2024-05-09T08:58:32.500573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"83'4","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606250+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:35.606250+0000","last_peered":"2024-05-09T08:58:35.606250+0000","last_clean":"2024-05-09T08:58:35.606250+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:35.606250+0000","last_undegraded":"2024-05-09T08:58:35.606250+0000","last_fullsized":"2024-05-09T08:58:35.606250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"83'3","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217416+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:33.217416+0000","last_peered":"2024-05-09T08:58:33.217416+0000","last_clean":"2024-05-09T08:58:33.217416+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:33.217416+0000","last_undegraded":"2024-05-09T08:58:33.217416+0000","last_fullsized":"2024-05-09T08:58:33.217416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.618711+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:35.618711+0000","last_peered":"2024-05-09T08:58:35.618711+0000","last_clean":"2024-05-09T08:58:35.618711+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:35.618711+0000","last_undegraded":"2024-05-09T08:58:35.618711+0000","last_fullsized":"2024-05-09T08:58:35.618711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395602+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395602+0000","last_peered":"2024-05-09T08:58:30.395602+0000","last_clean":"2024-05-09T08:58:30.395602+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395602+0000","last_undegraded":"2024-05-09T08:58:30.395602+0000","last_fullsized":"2024-05-09T08:58:30.395602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44595,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":89190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343248,"statfs":{"total":96636764160,"available":96607485952,"internally_reserved":0,"allocated":1609728,"data_stored":1050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1040384,"data_stored":495960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":461802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":459200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":496553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874247,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29800,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342040,"statfs":{"total":96636764160,"available":96606248960,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":459402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":1049112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:36.912 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:58:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.053+0000 7f50a5837640 1 Processor -- start 2024-05-09T08:58:37.052 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.053+0000 7f50a5837640 1 -- start start 2024-05-09T08:58:37.054 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.053+0000 7f50a5837640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f50a015b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f50a5837640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f50a006b7c0 0x7f50a006bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f50a5837640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a006c170 con 0x7f50a0155050 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f50a5837640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f50a006b330 con 0x7f50a006b7c0 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f50a5837640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f50a015e730 con 0x7f50a00c77d0 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509effd640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f50a0155050 0x7f50a006e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49706/0 (socket says 172.21.15.49:49706) 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509effd640 1 -- 172.21.15.49:0/2755567756 learned_addr learned my addr 172.21.15.49:0/2755567756 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509e7fc640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f50a015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960680955 0 0) 0x7f50a006c170 con 0x7f50a0155050 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f508c003520 con 0x7f50a0155050 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509f7fe640 1 --2- 172.21.15.49:0/2755567756 >> v2:172.21.15.183:3300/0 conn(0x7f50a006b7c0 0x7f50a006bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.055 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2943865616 0 0) 0x7f508c003520 con 0x7f50a0155050 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 msgr2=0x7f50a015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f50a015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 >> v2:172.21.15.183:3300/0 conn(0x7f50a006b7c0 msgr2=0x7f50a006bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 --2- 172.21.15.49:0/2755567756 >> v2:172.21.15.183:3300/0 conn(0x7f50a006b7c0 0x7f50a006bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a015ea10 con 0x7f50a0155050 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f50900019f0 con 0x7f50a0155050 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 >> v1:172.21.15.49:6789/0 conn(0x7f50a0155050 legacy=0x7f50a006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508c0039b0 0x7f508c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c004800 0x7f508c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f508c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50a006b330 con 0x7f508c0039b0 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50a015e730 con 0x7f508c004800 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f508c00b9f0 con 0x7f50a00c77d0 2024-05-09T08:58:37.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509effd640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508c0039b0 0x7f508c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.054+0000 7f509e7fc640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c004800 0x7f508c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509f7fe640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f508c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 -- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 msgr2=0x7f508c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a00c77d0 0x7f508c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 -- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c004800 msgr2=0x7f508c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508c004800 0x7f508c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 -- 172.21.15.49:0/2755567756 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c003520 con 0x7f508c0039b0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509effd640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508c0039b0 0x7f508c003dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f509000a7a0 tx=0x7f508c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba4b361e57691 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000a3f0 con 0x7f508c0039b0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509003b650 con 0x7f508c0039b0 2024-05-09T08:58:37.057 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.055+0000 7f509dffb640 1 -- 172.21.15.49:0/2755567756 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509003ba40 con 0x7f508c0039b0 2024-05-09T08:58:37.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 -- 172.21.15.49:0/2755567756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508c0039b0 msgr2=0x7f508c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 --2- 172.21.15.49:0/2755567756 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f508c0039b0 0x7f508c003dd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f509000a7a0 tx=0x7f508c005830 comp rx=0 tx=0).stop 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f509f7fe640 1 -- 172.21.15.49:0/2755567756 reap_dead start 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 -- 172.21.15.49:0/2755567756 shutdown_connections 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 -- 172.21.15.49:0/2755567756 >> 172.21.15.49:0/2755567756 conn(0x7f50a00c9620 msgr2=0x7f50a00c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 -- 172.21.15.49:0/2755567756 shutdown_connections 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.056+0000 7f50a5837640 1 -- 172.21.15.49:0/2755567756 wait complete. 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.057+0000 7f50a5837640 1 Processor -- start 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.057+0000 7f50a5837640 1 -- start start 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f508c0039b0 0x7f50a01eca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 0x7f50a0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f50a015f2b0 con 0x7f50a01452c0 2024-05-09T08:58:37.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50a015efd0 con 0x7f50a00ca7a0 2024-05-09T08:58:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f50a5837640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f50a015e3d0 con 0x7f508c0039b0 2024-05-09T08:58:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52866/0 (socket says 172.21.15.49:52866) 2024-05-09T08:58:37.062 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 -- 172.21.15.49:0/3942369254 learned_addr learned my addr 172.21.15.49:0/3942369254 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509effd640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f508c0039b0 0x7f50a01eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509e7fc640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 0x7f50a0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.064 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 -- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f508c0039b0 msgr2=0x7f50a01eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f508c0039b0 0x7f50a01eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 -- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 msgr2=0x7f50a0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 0x7f50a0071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.058+0000 7f509f7fe640 1 -- 172.21.15.49:0/3942369254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a00c3af0 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f509e7fc640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 0x7f50a0071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f509f7fe640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5098011490 tx=0x7f5098009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098003610 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509801b070 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a00cb7f0 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980039f0 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.059+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a00caf50 con 0x7f50a01452c0 2024-05-09T08:58:37.065 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.060+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a0068560 con 0x7f50a01452c0 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.060+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5098003b90 con 0x7f50a01452c0 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.061+0000 7f5097fff640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.061+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f509808b150 con 0x7f50a01452c0 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.061+0000 7f509effd640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.062+0000 7f509effd640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 0x7f508006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f509003b000 tx=0x7f5090009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:37.066 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.065+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50a00c3af0 con 0x7f50a01452c0 2024-05-09T08:58:37.252 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.253+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50a00c7d70 con 0x7f50a01452c0 2024-05-09T08:58:37.253 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.254+0000 7f5097fff640 1 -- 172.21.15.49:0/3942369254 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16838 (secure 0 0 0) 0x7f5098059a50 con 0x7f50a01452c0 2024-05-09T08:58:37.254 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:37.254 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":83,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:58:30.390193+0000","last_up_change":"2024-05-09T08:58:12.270880+0000","last_in_change":"2024-05-09T08:58:13.272774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":38,"up_from":65,"up_thru":82,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6809","nonce":282207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6811","nonce":282207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6815","nonce":282207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6813","nonce":282207482}]},"public_addr":"172.21.15.49:6809/282207482","cluster_addr":"172.21.15.49:6811/282207482","heartbeat_back_addr":"172.21.15.49:6815/282207482","heartbeat_front_addr":"172.21.15.49:6813/282207482","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":39},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":6}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 msgr2=0x7f508006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 0x7f508006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f509003b000 tx=0x7f5090009030 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 msgr2=0x7f50a0145740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5098011490 tx=0x7f5098009f90 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 shutdown_connections 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f508006be30 0x7f508006e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f50a01452c0 0x7f50a0145740 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a00ca7a0 0x7f50a0071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 --2- 172.21.15.49:0/3942369254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f508c0039b0 0x7f50a01eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 >> 172.21.15.49:0/3942369254 conn(0x7f50a00c9620 msgr2=0x7f50a00c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:37.256 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 shutdown_connections 2024-05-09T08:58:37.257 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:37.256+0000 7f50a5837640 1 -- 172.21.15.49:0/3942369254 wait complete. 2024-05-09T08:58:38.268 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T08:58:38.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.387+0000 7f9f95493640 1 Processor -- start 2024-05-09T08:58:38.387 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.387+0000 7f9f95493640 1 -- start start 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f95493640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f900cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f95493640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9f90072110 0x7f9f900c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f95493640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f900725a0 con 0x7f9f90071cf0 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f95493640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f9f90072880 con 0x7f9f90072110 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f95493640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9f90072b60 con 0x7f9f90155350 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f8e7fc640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f9f90071cf0 0x7f9f9015e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49710/0 (socket says 172.21.15.49:49710) 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.388+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/3413945186 learned_addr learned my addr 172.21.15.49:0/3413945186 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677463930 0 0) 0x7f9f900725a0 con 0x7f9f90071cf0 2024-05-09T08:58:38.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/3413945186 >> v2:172.21.15.183:3300/0 conn(0x7f9f90072110 0x7f9f900c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f7c003520 con 0x7f9f90071cf0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8effd640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f900cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2584170799 0 0) 0x7f9f7c003520 con 0x7f9f90071cf0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 msgr2=0x7f9f900cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f900cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 >> v2:172.21.15.183:3300/0 conn(0x7f9f90072110 msgr2=0x7f9f900c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 --2- 172.21.15.49:0/3413945186 >> v2:172.21.15.183:3300/0 conn(0x7f9f90072110 0x7f9f900c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f900c55a0 con 0x7f9f90071cf0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f9f78001670 con 0x7f9f90071cf0 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 >> v1:172.21.15.49:6789/0 conn(0x7f9f90071cf0 legacy=0x7f9f9015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7c003f30 0x7f9f7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90072880 con 0x7f9f7c003f30 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90072b60 con 0x7f9f7c004d40 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c00b9f0 con 0x7f9f90155350 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7c003f30 0x7f9f7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8effd640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.389+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 msgr2=0x7f9f7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f90155350 0x7f9f7c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c004d40 msgr2=0x7f9f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.393 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/3413945186 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c003520 con 0x7f9f7c003f30 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c004d40 0x7f9f7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7c003f30 0x7f9f7c004800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007fd0 tx=0x7f9f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9b32bf39d77989 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007d50 con 0x7f9f7c003f30 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78005ad0 con 0x7f9f7c003f30 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.390+0000 7f9f8dffb640 1 -- 172.21.15.49:0/3413945186 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007070 con 0x7f9f7c003f30 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 -- 172.21.15.49:0/3413945186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7c003f30 msgr2=0x7f9f7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 --2- 172.21.15.49:0/3413945186 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f7c003f30 0x7f9f7c004800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9f78007fd0 tx=0x7f9f7c005830 comp rx=0 tx=0).stop 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f8f7fe640 1 -- 172.21.15.49:0/3413945186 reap_dead start 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 -- 172.21.15.49:0/3413945186 shutdown_connections 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 -- 172.21.15.49:0/3413945186 >> 172.21.15.49:0/3413945186 conn(0x7f9f900ccfe0 msgr2=0x7f9f900cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.394 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 -- 172.21.15.49:0/3413945186 shutdown_connections 2024-05-09T08:58:38.395 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.391+0000 7f9f95493640 1 -- 172.21.15.49:0/3413945186 wait complete. 2024-05-09T08:58:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.392+0000 7f9f95493640 1 Processor -- start 2024-05-09T08:58:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 -- start start 2024-05-09T08:58:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c003f30 0x7f9f900ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 0x7f9f90156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.398 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f900cae40 0x7f9f901f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9015eb50 con 0x7f9f901f41a0 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90072e40 con 0x7f9f7c003f30 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f95493640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9015ecd0 con 0x7f9f900cae40 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 0x7f9f90156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f8e7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 0x7f9f90156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52886/0 (socket says 172.21.15.49:52886) 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.393+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/2531604182 learned_addr learned my addr 172.21.15.49:0/2531604182 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f900cae40 0x7f9f901f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8effd640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c003f30 0x7f9f900ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f900cae40 msgr2=0x7f9f901f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.399 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f900cae40 0x7f9f901f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c003f30 msgr2=0x7f9f900ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c003f30 0x7f9f900ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f900c89d0 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8effd640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c003f30 0x7f9f900ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f8e7fc640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 0x7f9f90156380 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c300 tx=0x7f9f900b4610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78042020 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f78004bf0 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78007070 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f900c8cb0 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.394+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f900c9270 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.395+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f901568c0 con 0x7f9f901f41a0 2024-05-09T08:58:38.400 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f78045020 con 0x7f9f901f41a0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9f7006be30 0x7f9f7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f9f780ab780 con 0x7f9f901f41a0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f9f70074290 0x7f9f70076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f7802be90 con 0x7f9f70074290 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f8effd640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9f7006be30 0x7f9f7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.396+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f9f780b2050 con 0x7f9f901f41a0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.397+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f9f70074290 0x7f9f70076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.397+0000 7f9f8f7fe640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f9f70074290 0x7f9f70076730 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.397+0000 7f9f8effd640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9f7006be30 0x7f9f7006e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9f880099e0 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.401 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.397+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== osd.0 v2:172.21.15.49:6800/1154213557 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f7802be90 con 0x7f9f70074290 2024-05-09T08:58:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 --> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9f9006d180 con 0x7f9f70074290 2024-05-09T08:58:38.410 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f877fe640 1 -- 172.21.15.49:0/2531604182 <== osd.0 v2:172.21.15.49:6800/1154213557 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9f9006d180 con 0x7f9f70074290 2024-05-09T08:58:38.410 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:38.410 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:38.411 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f9f70074290 msgr2=0x7f9f70076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:6800/1154213557,v1:172.21.15.49:6801/1154213557] conn(0x7f9f70074290 0x7f9f70076730 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9f7006be30 msgr2=0x7f9f7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f9f7006be30 0x7f9f7006e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9f880099e0 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 msgr2=0x7f9f90156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f95493640 1 --2- 172.21.15.49:0/2531604182 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f9f901f41a0 0x7f9f90156380 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c300 tx=0x7f9f900b4610 comp rx=0 tx=0).stop 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.411+0000 7f9f8f7fe640 1 -- 172.21.15.49:0/2531604182 reap_dead start 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.412+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 shutdown_connections 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.412+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 >> 172.21.15.49:0/2531604182 conn(0x7f9f900ccfe0 msgr2=0x7f9f900ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.412+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 shutdown_connections 2024-05-09T08:58:38.412 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.412+0000 7f9f95493640 1 -- 172.21.15.49:0/2531604182 wait complete. 2024-05-09T08:58:38.423 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T08:58:38.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.528+0000 7f47e0c2c640 1 Processor -- start 2024-05-09T08:58:38.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.528+0000 7f47e0c2c640 1 -- start start 2024-05-09T08:58:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.529+0000 7f47e0c2c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47dc154b90 0x7f47dc0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.529+0000 7f47e0c2c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47e0c2c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47dc072600 con 0x7f47dc0c6640 2024-05-09T08:58:38.531 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47e0c2c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f47dc0728e0 con 0x7f47dc154b90 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47e0c2c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc0c5880 con 0x7f47dc154f70 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47dc154b90 0x7f47dc0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f47dc154b90 0x7f47dc0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52892/0 (socket says 172.21.15.49:52892) 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 -- 172.21.15.49:0/1824830617 learned_addr learned my addr 172.21.15.49:0/1824830617 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47d9d9b640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 -- 172.21.15.49:0/1824830617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 msgr2=0x7f47dc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 -- 172.21.15.49:0/1824830617 >> v1:172.21.15.49:6789/0 conn(0x7f47dc0c6640 legacy=0x7f47dc0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.530+0000 7f47da59c640 1 -- 172.21.15.49:0/1824830617 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc0c5b60 con 0x7f47dc154b90 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47da59c640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc154b90 0x7f47dc0cb280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009870 tx=0x7f47cc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80608b348362eef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47d959a640 1 -- 172.21.15.49:0/1824830617 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc037070 con 0x7f47dc154b90 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47d959a640 1 -- 172.21.15.49:0/1824830617 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc02ae50 con 0x7f47dc154b90 2024-05-09T08:58:38.532 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47d959a640 1 -- 172.21.15.49:0/1824830617 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc032860 con 0x7f47dc154b90 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 -- 172.21.15.49:0/1824830617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc154b90 msgr2=0x7f47dc0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc154b90 0x7f47dc0cb280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009870 tx=0x7f47cc02a300 comp rx=0 tx=0).stop 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 -- 172.21.15.49:0/1824830617 shutdown_connections 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/1824830617 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc154b90 0x7f47dc0cb280 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.531+0000 7f47e0c2c640 1 -- 172.21.15.49:0/1824830617 >> 172.21.15.49:0/1824830617 conn(0x7f47dc074840 msgr2=0x7f47dc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.532+0000 7f47e0c2c640 1 -- 172.21.15.49:0/1824830617 shutdown_connections 2024-05-09T08:58:38.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.532+0000 7f47e0c2c640 1 -- 172.21.15.49:0/1824830617 wait complete. 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.533+0000 7f47e0c2c640 1 Processor -- start 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.533+0000 7f47e0c2c640 1 -- start start 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc154b90 0x7f47dc0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc15af40 con 0x7f47dc0c6640 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47dc0c6120 con 0x7f47dc154b90 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f47dc071820 con 0x7f47dc154f70 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.536 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52900/0 (socket says 172.21.15.49:52900) 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 -- 172.21.15.49:0/4242496638 learned_addr learned my addr 172.21.15.49:0/4242496638 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47dad9d640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47d9d9b640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc154b90 0x7f47dc0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 -- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 msgr2=0x7f47dc159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 -- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc154b90 msgr2=0x7f47dc0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc154b90 0x7f47dc0d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 -- 172.21.15.49:0/4242496638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47dc1f14c0 con 0x7f47dc0c6640 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47dad9d640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47da59c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc041070 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47cc03c670 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc037040 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47dc1f1780 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.534+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47dc0cf640 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.535+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47dc0b3b80 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.536+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47cc044070 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.536+0000 7f47caffd640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 0x7f47bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.537+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f47cc0aba00 con 0x7f47dc0c6640 2024-05-09T08:58:38.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.537+0000 7f47d9d9b640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 0x7f47bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.539 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.537+0000 7f47d9d9b640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 0x7f47bc06e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f47d0006fd0 tx=0x7f47d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.540 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.541+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f47dc0b3b80 con 0x7f47dc0c6640 2024-05-09T08:58:38.723 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.724+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f47dc04f660 con 0x7f47bc06be30 2024-05-09T08:58:38.724 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.725+0000 7f47caffd640 1 -- 172.21.15.49:0/4242496638 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f47dc04f660 con 0x7f47bc06be30 2024-05-09T08:58:38.724 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T08:58:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 msgr2=0x7f47bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 0x7f47bc06e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f47d0006fd0 tx=0x7f47d0008040 comp rx=0 tx=0).stop 2024-05-09T08:58:38.726 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 msgr2=0x7f47dc0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 shutdown_connections 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47bc06be30 0x7f47bc06e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47dc154f70 0x7f47dc159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47dc154b90 0x7f47dc0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 --2- 172.21.15.49:0/4242496638 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f47dc0c6640 0x7f47dc0ce390 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 >> 172.21.15.49:0/4242496638 conn(0x7f47dc074840 msgr2=0x7f47dc0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 shutdown_connections 2024-05-09T08:58:38.727 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.727+0000 7f47e0c2c640 1 -- 172.21.15.49:0/4242496638 wait complete. 2024-05-09T08:58:38.737 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T08:58:38.737 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.846+0000 7fbbc428c640 1 Processor -- start 2024-05-09T08:58:38.845 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.846+0000 7fbbc428c640 1 -- start start 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc428c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc428c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 0x7fbbbc0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc428c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbbc0c2550 con 0x7fbbbc152b60 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc428c640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fbbbc0c2b30 con 0x7fbbbc0c9ab0 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc428c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fbbbc0c3500 con 0x7fbbbc0c78f0 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52902/0 (socket says 172.21.15.49:52902) 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 -- 172.21.15.49:0/366125507 learned_addr learned my addr 172.21.15.49:0/366125507 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc2829640 1 --2- 172.21.15.49:0/366125507 >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 0x7fbbbc0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.847 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 -- 172.21.15.49:0/366125507 >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 msgr2=0x7fbbbc0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 --2- 172.21.15.49:0/366125507 >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 0x7fbbbc0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 -- 172.21.15.49:0/366125507 >> v1:172.21.15.49:6789/0 conn(0x7fbbbc152b60 legacy=0x7fbbbc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc1827640 1 -- 172.21.15.49:0/366125507 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc0c0820 con 0x7fbbbc0c9ab0 2024-05-09T08:58:38.848 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.847+0000 7fbbc2829640 1 --2- 172.21.15.49:0/366125507 >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 0x7fbbbc0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.848+0000 7fbbc1827640 1 --2- 172.21.15.49:0/366125507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc075780 tx=0x7fbbac02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a7e82758586c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.848+0000 7fbbc1026640 1 -- 172.21.15.49:0/366125507 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac037070 con 0x7fbbbc0c9ab0 2024-05-09T08:58:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.848+0000 7fbbc1026640 1 -- 172.21.15.49:0/366125507 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac02ae50 con 0x7fbbbc0c9ab0 2024-05-09T08:58:38.849 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.848+0000 7fbbc1026640 1 -- 172.21.15.49:0/366125507 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac032860 con 0x7fbbbc0c9ab0 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 -- 172.21.15.49:0/366125507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc0c9ab0 msgr2=0x7fbbbc0c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 --2- 172.21.15.49:0/366125507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc075780 tx=0x7fbbac02a300 comp rx=0 tx=0).stop 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 -- 172.21.15.49:0/366125507 shutdown_connections 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 --2- 172.21.15.49:0/366125507 >> v2:172.21.15.183:3300/0 conn(0x7fbbbc0c78f0 0x7fbbbc0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 --2- 172.21.15.49:0/366125507 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc0c9ab0 0x7fbbbc0c2fc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 -- 172.21.15.49:0/366125507 >> 172.21.15.49:0/366125507 conn(0x7fbbbc0c8e00 msgr2=0x7fbbbc0c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 -- 172.21.15.49:0/366125507 shutdown_connections 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.849+0000 7fbbc428c640 1 -- 172.21.15.49:0/366125507 wait complete. 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.850+0000 7fbbc428c640 1 Processor -- start 2024-05-09T08:58:38.850 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.850+0000 7fbbc428c640 1 -- start start 2024-05-09T08:58:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbbc0c78f0 0x7fbbbc15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 0x7fbbbc0c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbbc0be330 0x7fbbbc0be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc152f40 con 0x7fbbbc152b60 2024-05-09T08:58:38.853 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbbbc0c27c0 con 0x7fbbbc0be330 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc428c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbbbc1530c0 con 0x7fbbbc0c78f0 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 0x7fbbbc0c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 0x7fbbbc0c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52908/0 (socket says 172.21.15.49:52908) 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 -- 172.21.15.49:0/3989349684 learned_addr learned my addr 172.21.15.49:0/3989349684 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc2028640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbbc0c78f0 0x7fbbbc15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc2829640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbbc0be330 0x7fbbbc0be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 -- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbbc0c78f0 msgr2=0x7fbbbc15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbbc0c78f0 0x7fbbbc15c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 -- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbbc0be330 msgr2=0x7fbbbc0be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbbc0be330 0x7fbbbc0be7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbbc1e4a20 con 0x7fbbbc152b60 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc2028640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbbbc0c78f0 0x7fbbbc15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.854 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc2829640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbbbc0be330 0x7fbbbc0be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.851+0000 7fbbc1827640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 0x7fbbbc0c40e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02a340 tx=0x7fbbac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac041020 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac03c870 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbbc0bed00 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac037040 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbbc1f14d0 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.852+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbbc15ca00 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.853+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbac044020 con 0x7fbbbc152b60 2024-05-09T08:58:38.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbb2ffd640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbba006be30 0x7fbba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fbbac0ab830 con 0x7fbbbc152b60 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbc2028640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbba006be30 0x7fbba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbb2ffd640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbba0074290 0x7fbba0076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbba0076e00 con 0x7fbba0074290 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbbac0b02c0 con 0x7fbbbc152b60 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbc2829640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbba0074290 0x7fbba0076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbc2829640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbba0074290 0x7fbba0076730 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.854+0000 7fbbc2028640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbba006be30 0x7fbba006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80015e0 tx=0x7fbbb8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.855+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== osd.1 v2:172.21.15.49:6816/3346547076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbba0076e00 con 0x7fbba0074290 2024-05-09T08:58:38.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.868+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 --> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbbbc068560 con 0x7fbba0074290 2024-05-09T08:58:38.868 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbb2ffd640 1 -- 172.21.15.49:0/3989349684 <== osd.1 v2:172.21.15.49:6816/3346547076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbbbc068560 con 0x7fbba0074290 2024-05-09T08:58:38.868 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:38.868 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:38.868 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbba0074290 msgr2=0x7fbba0076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:6816/3346547076,v1:172.21.15.49:6817/3346547076] conn(0x7fbba0074290 0x7fbba0076730 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbba006be30 msgr2=0x7fbba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbba006be30 0x7fbba006e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80015e0 tx=0x7fbbb8009040 comp rx=0 tx=0).stop 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 msgr2=0x7fbbbc0c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.869 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 --2- 172.21.15.49:0/3989349684 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbbbc152b60 0x7fbbbc0c40e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02a340 tx=0x7fbbac0046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc2829640 1 -- 172.21.15.49:0/3989349684 reap_dead start 2024-05-09T08:58:38.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 shutdown_connections 2024-05-09T08:58:38.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 >> 172.21.15.49:0/3989349684 conn(0x7fbbbc0c8e00 msgr2=0x7fbbbc14fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 shutdown_connections 2024-05-09T08:58:38.870 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.869+0000 7fbbc428c640 1 -- 172.21.15.49:0/3989349684 wait complete. 2024-05-09T08:58:38.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.984+0000 7f7b98928640 1 Processor -- start 2024-05-09T08:58:38.983 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.984+0000 7f7b98928640 1 -- start start 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b98928640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b94151be0 0x7f7b940c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b98928640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 0x7f7b9406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b98928640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b940c4b10 con 0x7f7b9415af60 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b98928640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f7b940c5c70 con 0x7f7b94151be0 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b98928640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f7b940c4da0 con 0x7f7b940c73a0 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b9259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b94151be0 0x7f7b940c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b9259c640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f7b94151be0 0x7f7b940c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52910/0 (socket says 172.21.15.49:52910) 2024-05-09T08:58:38.985 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b9259c640 1 -- 172.21.15.49:0/599844679 learned_addr learned my addr 172.21.15.49:0/599844679 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.985+0000 7f7b92d9d640 1 --2- 172.21.15.49:0/599844679 >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 0x7f7b9406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9259c640 1 -- 172.21.15.49:0/599844679 >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 msgr2=0x7f7b9406dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9259c640 1 --2- 172.21.15.49:0/599844679 >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 0x7f7b9406dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9259c640 1 -- 172.21.15.49:0/599844679 >> v1:172.21.15.49:6789/0 conn(0x7f7b9415af60 legacy=0x7f7b940c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9259c640 1 -- 172.21.15.49:0/599844679 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940c5060 con 0x7f7b94151be0 2024-05-09T08:58:38.986 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b92d9d640 1 --2- 172.21.15.49:0/599844679 >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 0x7f7b9406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9259c640 1 --2- 172.21.15.49:0/599844679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b94151be0 0x7f7b940c45d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406b290 tx=0x7f7b8c02c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e6815a36d5c85f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.987 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9159a640 1 -- 172.21.15.49:0/599844679 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c035020 con 0x7f7b94151be0 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9159a640 1 -- 172.21.15.49:0/599844679 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c031620 con 0x7f7b94151be0 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.986+0000 7f7b9159a640 1 -- 172.21.15.49:0/599844679 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0319f0 con 0x7f7b94151be0 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 -- 172.21.15.49:0/599844679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b94151be0 msgr2=0x7f7b940c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 --2- 172.21.15.49:0/599844679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b94151be0 0x7f7b940c45d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406b290 tx=0x7f7b8c02c9d0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 -- 172.21.15.49:0/599844679 shutdown_connections 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 --2- 172.21.15.49:0/599844679 >> v2:172.21.15.183:3300/0 conn(0x7f7b940c73a0 0x7f7b9406dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 --2- 172.21.15.49:0/599844679 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b94151be0 0x7f7b940c45d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 -- 172.21.15.49:0/599844679 >> 172.21.15.49:0/599844679 conn(0x7f7b940cc290 msgr2=0x7f7b940cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 -- 172.21.15.49:0/599844679 shutdown_connections 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.987+0000 7f7b98928640 1 -- 172.21.15.49:0/599844679 wait complete. 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.988+0000 7f7b98928640 1 Processor -- start 2024-05-09T08:58:38.988 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 -- start start 2024-05-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 0x7f7b941f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 0x7f7b940cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940c5900 con 0x7f7b941f0a40 2024-05-09T08:58:38.991 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b940c5620 con 0x7f7b9415af60 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b98928640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f7b94151fc0 con 0x7f7b94151be0 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b92d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b92d9d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52914/0 (socket says 172.21.15.49:52914) 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.989+0000 7f7b92d9d640 1 -- 172.21.15.49:0/2549419486 learned_addr learned my addr 172.21.15.49:0/2549419486 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b9259c640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 0x7f7b941f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 -- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 msgr2=0x7f7b941f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b91d9b640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 0x7f7b940cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 0x7f7b941f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 -- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 msgr2=0x7f7b940cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 0x7f7b940cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 -- 172.21.15.49:0/2549419486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b940cbd10 con 0x7f7b941f0a40 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b9259c640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 0x7f7b941f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.992 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b91d9b640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 0x7f7b940cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b92d9d640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d5f0 tx=0x7f7b8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84004030 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b840041d0 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b941e3030 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84003610 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.990+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941e32f0 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.991+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9404f660 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.992+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b840037b0 con 0x7f7b941f0a40 2024-05-09T08:58:38.993 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.992+0000 7f7b8affd640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:38.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.992+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f7b8408b0a0 con 0x7f7b941f0a40 2024-05-09T08:58:38.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.992+0000 7f7b9259c640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:38.994 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.993+0000 7f7b9259c640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 0x7f7b7406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:38.996 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:38.997+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7b84059a30 con 0x7f7b941f0a40 2024-05-09T08:58:39.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.183+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7b941f4550 con 0x7f7b7406be30 2024-05-09T08:58:39.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.184+0000 7f7b8affd640 1 -- 172.21.15.49:0/2549419486 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7b8c03a420 con 0x7f7b7406be30 2024-05-09T08:58:39.183 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 msgr2=0x7f7b7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 0x7f7b7406e2f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c036000 tx=0x7f7b8c009000 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 msgr2=0x7f7b941e0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400d5f0 tx=0x7f7b8400dab0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 shutdown_connections 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f7b7406be30 0x7f7b7406e2f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f7b941f0a40 0x7f7b941e0da0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b9415af60 0x7f7b940cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 --2- 172.21.15.49:0/2549419486 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b94151be0 0x7f7b941f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 >> 172.21.15.49:0/2549419486 conn(0x7f7b940cc290 msgr2=0x7f7b94072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 shutdown_connections 2024-05-09T08:58:39.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.186+0000 7f7b98928640 1 -- 172.21.15.49:0/2549419486 wait complete. 2024-05-09T08:58:39.196 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T08:58:39.197 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.306+0000 7fac235a5640 1 Processor -- start 2024-05-09T08:58:39.305 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.306+0000 7fac235a5640 1 -- start start 2024-05-09T08:58:39.308 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac235a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac1c071a00 0x7fac1c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac235a5640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c0c96b0 con 0x7fac1c152e50 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac235a5640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c067d40 con 0x7fac1c073430 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac235a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c0c8980 con 0x7fac1c071a00 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac21341640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fac1c152e50 0x7fac1c15c0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49754/0 (socket says 172.21.15.49:49754) 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac21341640 1 -- 172.21.15.49:0/1287474393 learned_addr learned my addr 172.21.15.49:0/1287474393 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac21b42640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac1c071a00 0x7fac1c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3626286306 0 0) 0x7fac1c0c96b0 con 0x7fac1c152e50 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac08003520 con 0x7fac1c152e50 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3545525165 0 0) 0x7fac08003520 con 0x7fac1c152e50 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac1c071a00 msgr2=0x7fac1c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac1c071a00 0x7fac1c0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 >> v1:172.21.15.183:6789/0 conn(0x7fac1c073430 legacy=0x7fac1c0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c070a60 con 0x7fac1c152e50 2024-05-09T08:58:39.309 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fac18001670 con 0x7fac1c152e50 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 >> v1:172.21.15.49:6789/0 conn(0x7fac1c152e50 legacy=0x7fac1c15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.307+0000 7fac13fff640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac080039e0 0x7fac08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac13fff640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080043b0 0x7fac08004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac13fff640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08004fa0 0x7fac0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0c8980 con 0x7fac080039e0 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac0800bf00 con 0x7fac08004fa0 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fac0800c080 con 0x7fac080043b0 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac080039e0 0x7fac08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 -- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080043b0 msgr2=0x7fac08004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080043b0 0x7fac08004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 -- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08004fa0 msgr2=0x7fac0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac08004fa0 0x7fac0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.310 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.308+0000 7fac21341640 1 -- 172.21.15.49:0/1287474393 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac08003520 con 0x7fac080039e0 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac21341640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac080039e0 0x7fac08003de0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac1800c240 tx=0x7fac08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbcb895ff03e8a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18007d10 con 0x7fac080039e0 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac18005ad0 con 0x7fac080039e0 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac13fff640 1 -- 172.21.15.49:0/1287474393 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18007070 con 0x7fac080039e0 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 -- 172.21.15.49:0/1287474393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac080039e0 msgr2=0x7fac08003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 --2- 172.21.15.49:0/1287474393 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac080039e0 0x7fac08003de0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fac1800c240 tx=0x7fac08005830 comp rx=0 tx=0).stop 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac21b42640 1 -- 172.21.15.49:0/1287474393 reap_dead start 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 -- 172.21.15.49:0/1287474393 shutdown_connections 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 -- 172.21.15.49:0/1287474393 >> 172.21.15.49:0/1287474393 conn(0x7fac1c0cce00 msgr2=0x7fac1c0cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 -- 172.21.15.49:0/1287474393 shutdown_connections 2024-05-09T08:58:39.311 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.309+0000 7fac235a5640 1 -- 172.21.15.49:0/1287474393 wait complete. 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 Processor -- start 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 -- start start 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080039e0 0x7fac1c153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac1c0c41b0 0x7fac1c14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 0x7fac1c1ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0c8bd0 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.315 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c071000 con 0x7fac1c0c41b0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.311+0000 7fac235a5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c0c8eb0 con 0x7fac080039e0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 0x7fac1c1ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 0x7fac1c1ee920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52938/0 (socket says 172.21.15.49:52938) 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 -- 172.21.15.49:0/3785140134 learned_addr learned my addr 172.21.15.49:0/3785140134 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21341640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080039e0 0x7fac1c153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac20b40640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac1c0c41b0 0x7fac1c14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 -- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080039e0 msgr2=0x7fac1c153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fac080039e0 0x7fac1c153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 -- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac1c0c41b0 msgr2=0x7fac1c14c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac1c0c41b0 0x7fac1c14c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1eee60 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.316 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac20b40640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac1c0c41b0 0x7fac1c14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac21b42640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 0x7fac1c1ee920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fac1c06de20 tx=0x7fac1400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1400bce0 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac1401a070 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c152020 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14014640 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.312+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1522c0 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.313+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac1c1ee050 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.314+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac14005d10 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.314+0000 7fac11ffb640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fac0006be30 0x7fac0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.314+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fac1408af10 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.317 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac21341640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fac0006be30 0x7fac0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac11ffb640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7fac00074290 0x7fac00076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac00076e00 con 0x7fac00074290 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fac1408d610 con 0x7fac1c1ee4a0 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac20b40640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7fac00074290 0x7fac00076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac20b40640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7fac00074290 0x7fac00076730 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.315+0000 7fac21341640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fac0006be30 0x7fac0006e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac18033000 tx=0x7fac18009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.318 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.316+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== osd.2 v2:172.21.15.49:6808/282207482 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac00076e00 con 0x7fac00074290 2024-05-09T08:58:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 --> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fac1c0c7e50 con 0x7fac00074290 2024-05-09T08:58:39.328 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac11ffb640 1 -- 172.21.15.49:0/3785140134 <== osd.2 v2:172.21.15.49:6808/282207482 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7fac1c0c7e50 con 0x7fac00074290 2024-05-09T08:58:39.328 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:39.328 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-09T08:58:39.329 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7fac00074290 msgr2=0x7fac00076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:6808/282207482,v1:172.21.15.49:6809/282207482] conn(0x7fac00074290 0x7fac00076730 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fac0006be30 msgr2=0x7fac0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fac0006be30 0x7fac0006e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fac18033000 tx=0x7fac18009030 comp rx=0 tx=0).stop 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 msgr2=0x7fac1c1ee920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 --2- 172.21.15.49:0/3785140134 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fac1c1ee4a0 0x7fac1c1ee920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fac1c06de20 tx=0x7fac1400b480 comp rx=0 tx=0).stop 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac21b42640 1 -- 172.21.15.49:0/3785140134 reap_dead start 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 shutdown_connections 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.329+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 >> 172.21.15.49:0/3785140134 conn(0x7fac1c0cce00 msgr2=0x7fac1c0712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.330+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 shutdown_connections 2024-05-09T08:58:39.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.330+0000 7fac235a5640 1 -- 172.21.15.49:0/3785140134 wait complete. 2024-05-09T08:58:39.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T08:58:39.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 Processor -- start 2024-05-09T08:58:39.444 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 -- start start 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f89ac155180 0x7f89ac0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f89ac0c8f90 0x7f89ac06b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac06b740 0x7f89ac0c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f89ac0c7b90 con 0x7f89ac0c8f90 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f89ac15f160 con 0x7f89ac155180 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.445+0000 7f89b0bb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac0c78b0 con 0x7f89ac06b740 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f89ac0c8f90 0x7f89ac06b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.446 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f89ac0c8f90 0x7f89ac06b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52950/0 (socket says 172.21.15.49:52950) 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 -- 172.21.15.49:0/978346445 learned_addr learned my addr 172.21.15.49:0/978346445 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89aad9d640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac06b740 0x7f89ac0c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89aa59c640 1 --2- 172.21.15.49:0/978346445 >> v2:172.21.15.183:3300/0 conn(0x7f89ac155180 0x7f89ac0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 -- 172.21.15.49:0/978346445 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac06b740 msgr2=0x7f89ac0c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac06b740 0x7f89ac0c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 -- 172.21.15.49:0/978346445 >> v2:172.21.15.183:3300/0 conn(0x7f89ac155180 msgr2=0x7f89ac0c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 --2- 172.21.15.49:0/978346445 >> v2:172.21.15.183:3300/0 conn(0x7f89ac155180 0x7f89ac0c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.446+0000 7f89a3fff640 1 -- 172.21.15.49:0/978346445 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac15e3a0 con 0x7f89ac0c8f90 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89a3fff640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac06b200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f89ac06cef0 tx=0x7f899402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f9a97211e2112f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89a9d9b640 1 -- 172.21.15.49:0/978346445 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994037070 con 0x7f89ac0c8f90 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89a9d9b640 1 -- 172.21.15.49:0/978346445 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8994032440 con 0x7f89ac0c8f90 2024-05-09T08:58:39.447 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89a9d9b640 1 -- 172.21.15.49:0/978346445 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994032810 con 0x7f89ac0c8f90 2024-05-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 -- 172.21.15.49:0/978346445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 msgr2=0x7f89ac06b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac06b200 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f89ac06cef0 tx=0x7f899402a140 comp rx=0 tx=0).stop 2024-05-09T08:58:39.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 -- 172.21.15.49:0/978346445 shutdown_connections 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac06b740 0x7f89ac0c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/978346445 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac06b200 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/978346445 >> v2:172.21.15.183:3300/0 conn(0x7f89ac155180 0x7f89ac0c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 -- 172.21.15.49:0/978346445 >> 172.21.15.49:0/978346445 conn(0x7f89ac0c9650 msgr2=0x7f89ac0c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 -- 172.21.15.49:0/978346445 shutdown_connections 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.447+0000 7f89b0bb6640 1 -- 172.21.15.49:0/978346445 wait complete. 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 Processor -- start 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 -- start start 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89ac06b740 0x7f89ac14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac155180 0x7f89ac06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac0c7550 con 0x7f89ac0c8f90 2024-05-09T08:58:39.451 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89ac15e960 con 0x7f89ac06b740 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.449+0000 7f89b0bb6640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f89ac0c32a0 con 0x7f89ac155180 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52958/0 (socket says 172.21.15.49:52958) 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 -- 172.21.15.49:0/1032793200 learned_addr learned my addr 172.21.15.49:0/1032793200 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89aad9d640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac155180 0x7f89ac06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89aa59c640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89ac06b740 0x7f89ac14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 -- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac155180 msgr2=0x7f89ac06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac155180 0x7f89ac06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 -- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89ac06b740 msgr2=0x7f89ac14e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89ac06b740 0x7f89ac14e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 -- 172.21.15.49:0/1032793200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ac1f0820 con 0x7f89ac0c8f90 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a3fff640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f899402a180 tx=0x7f89940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.452 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994041020 con 0x7f89ac0c8f90 2024-05-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89940306c0 con 0x7f89ac0c8f90 2024-05-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ac071550 con 0x7f89ac0c8f90 2024-05-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994037040 con 0x7f89ac0c8f90 2024-05-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.450+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ac158210 con 0x7f89ac0c8f90 2024-05-09T08:58:39.453 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.451+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ac04f660 con 0x7f89ac0c8f90 2024-05-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.452+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8994044020 con 0x7f89ac0c8f90 2024-05-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.452+0000 7f89a2ffd640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 0x7f898c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.453+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f89940ab4e0 con 0x7f89ac0c8f90 2024-05-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.453+0000 7f89aa59c640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 0x7f898c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.454 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.453+0000 7f89aa59c640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 0x7f898c06e320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f899c00aa20 tx=0x7f899c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.456 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.457+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f899404b030 con 0x7f89ac0c8f90 2024-05-09T08:58:39.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.640+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f89ac0c7d70 con 0x7f898c06be60 2024-05-09T08:58:39.639 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.640+0000 7f89a2ffd640 1 -- 172.21.15.49:0/1032793200 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f89ac0c7d70 con 0x7f898c06be60 2024-05-09T08:58:39.639 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 msgr2=0x7f898c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 0x7f898c06e320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f899c00aa20 tx=0x7f899c008040 comp rx=0 tx=0).stop 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 msgr2=0x7f89ac072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f899402a180 tx=0x7f89940046c0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 shutdown_connections 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f898c06be60 0x7f898c06e320 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f89ac155180 0x7f89ac06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.642 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f89ac0c8f90 0x7f89ac072030 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 --2- 172.21.15.49:0/1032793200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89ac06b740 0x7f89ac14e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 >> 172.21.15.49:0/1032793200 conn(0x7f89ac0c9650 msgr2=0x7f89ac0cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 shutdown_connections 2024-05-09T08:58:39.643 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.642+0000 7f89b0bb6640 1 -- 172.21.15.49:0/1032793200 wait complete. 2024-05-09T08:58:39.653 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T08:58:39.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:39.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.762+0000 7f5891bc1640 1 Processor -- start 2024-05-09T08:58:39.761 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.762+0000 7f5891bc1640 1 -- start start 2024-05-09T08:58:39.763 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f5891bc1640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f588c154f40 0x7f588c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f5891bc1640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f588c15ed60 0x7f588c1654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f5891bc1640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c0c8bd0 con 0x7f588c073580 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f5891bc1640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f588c0c7800 con 0x7f588c15ed60 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f5891bc1640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f588c15b730 con 0x7f588c154f40 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f588c15ed60 0x7f588c1654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f588c15ed60 0x7f588c1654b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52960/0 (socket says 172.21.15.49:52960) 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 -- 172.21.15.49:0/235297970 learned_addr learned my addr 172.21.15.49:0/235297970 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588b7fe640 1 --2- 172.21.15.49:0/235297970 >> v2:172.21.15.183:3300/0 conn(0x7f588c154f40 0x7f588c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 -- 172.21.15.49:0/235297970 >> v2:172.21.15.183:3300/0 conn(0x7f588c154f40 msgr2=0x7f588c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 --2- 172.21.15.49:0/235297970 >> v2:172.21.15.183:3300/0 conn(0x7f588c154f40 0x7f588c15e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 -- 172.21.15.49:0/235297970 >> v1:172.21.15.183:6789/0 conn(0x7f588c073580 legacy=0x7f588c071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.764 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.763+0000 7f588bfff640 1 -- 172.21.15.49:0/235297970 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c15b9f0 con 0x7f588c15ed60 2024-05-09T08:58:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.764+0000 7f588bfff640 1 --2- 172.21.15.49:0/235297970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c1654b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f588c075850 tx=0x7f588402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ef25e35987c71e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.764+0000 7f588a7fc640 1 -- 172.21.15.49:0/235297970 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884036070 con 0x7f588c15ed60 2024-05-09T08:58:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.764+0000 7f588a7fc640 1 -- 172.21.15.49:0/235297970 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884031600 con 0x7f588c15ed60 2024-05-09T08:58:39.765 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.764+0000 7f588a7fc640 1 -- 172.21.15.49:0/235297970 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840319f0 con 0x7f588c15ed60 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 -- 172.21.15.49:0/235297970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 msgr2=0x7f588c1654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 --2- 172.21.15.49:0/235297970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c1654b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f588c075850 tx=0x7f588402a7c0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 -- 172.21.15.49:0/235297970 shutdown_connections 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 --2- 172.21.15.49:0/235297970 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c1654b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 --2- 172.21.15.49:0/235297970 >> v2:172.21.15.183:3300/0 conn(0x7f588c154f40 0x7f588c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 -- 172.21.15.49:0/235297970 >> 172.21.15.49:0/235297970 conn(0x7f588c06b200 msgr2=0x7f588c06b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.768 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 -- 172.21.15.49:0/235297970 shutdown_connections 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.765+0000 7f5891bc1640 1 -- 172.21.15.49:0/235297970 wait complete. 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.766+0000 7f5891bc1640 1 Processor -- start 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.766+0000 7f5891bc1640 1 -- start start 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f588c073580 0x7f588c14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f588c154f40 0x7f588c1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f588c15e300 con 0x7f588c15ed60 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f588c06f720 con 0x7f588c073580 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f5891bc1640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f588c071b80 con 0x7f588c154f40 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f588bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f588bfff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52964/0 (socket says 172.21.15.49:52964) 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f588bfff640 1 -- 172.21.15.49:0/2409538339 learned_addr learned my addr 172.21.15.49:0/2409538339 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.769 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f588b7fe640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f588c073580 0x7f588c14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.767+0000 7f588affd640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f588c154f40 0x7f588c1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 -- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f588c154f40 msgr2=0x7f588c1f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f588c154f40 0x7f588c1f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 -- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f588c073580 msgr2=0x7f588c14e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f588c073580 0x7f588c14e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c14ea80 con 0x7f588c15ed60 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f588bfff640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c0c8dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5884009f40 tx=0x7f5884004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884036050 con 0x7f588c15ed60 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884037040 con 0x7f588c15ed60 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c154300 con 0x7f588c15ed60 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884009510 con 0x7f588c15ed60 2024-05-09T08:58:39.770 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.768+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1527d0 con 0x7f588c15ed60 2024-05-09T08:58:39.771 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.769+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f588c153170 con 0x7f588c15ed60 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5884044020 con 0x7f588c15ed60 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f587806be30 0x7f587806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f58840ab8c0 con 0x7f588c15ed60 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5878074290 0x7f5878076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f588404ce50 con 0x7f5878074290 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f58840abd50 con 0x7f588c15ed60 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f588b7fe640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f587806be30 0x7f587806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.770+0000 7f588affd640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5878074290 0x7f5878076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.772 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.771+0000 7f588b7fe640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f587806be30 0x7f587806e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f587c009ad0 tx=0x7f587c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.771+0000 7f588affd640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5878074290 0x7f5878076730 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.773 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.771+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== osd.3 v2:172.21.15.49:6822/1536339711 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f588404ce50 con 0x7f5878074290 2024-05-09T08:58:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 --> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f588c075b60 con 0x7f5878074290 2024-05-09T08:58:39.784 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5873fff640 1 -- 172.21.15.49:0/2409538339 <== osd.3 v2:172.21.15.49:6822/1536339711 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f588c075b60 con 0x7f5878074290 2024-05-09T08:58:39.784 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:39.784 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:39.784 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5878074290 msgr2=0x7f5878076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:6822/1536339711,v1:172.21.15.49:6824/1536339711] conn(0x7f5878074290 0x7f5878076730 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f587806be30 msgr2=0x7f587806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f587806be30 0x7f587806e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f587c009ad0 tx=0x7f587c008040 comp rx=0 tx=0).stop 2024-05-09T08:58:39.785 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 msgr2=0x7f588c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 --2- 172.21.15.49:0/2409538339 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f588c15ed60 0x7f588c0c8dd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5884009f40 tx=0x7f5884004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f588bfff640 1 -- 172.21.15.49:0/2409538339 reap_dead start 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 shutdown_connections 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 >> 172.21.15.49:0/2409538339 conn(0x7f588c06b200 msgr2=0x7f588c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 shutdown_connections 2024-05-09T08:58:39.786 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.785+0000 7f5891bc1640 1 -- 172.21.15.49:0/2409538339 wait complete. 2024-05-09T08:58:39.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T08:58:39.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.901+0000 7f492a7ce640 1 Processor -- start 2024-05-09T08:58:39.900 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.901+0000 7f492a7ce640 1 -- start start 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.901+0000 7f492a7ce640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4924154b90 0x7f49240cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.901+0000 7f492a7ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154f70 0x7f49240cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f492a7ce640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4924072600 con 0x7f49240c6640 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f492a7ce640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f49240728e0 con 0x7f4924154b90 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f492a7ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f49240c5880 con 0x7f4924154f70 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4924154b90 0x7f49240cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f4924154b90 0x7f49240cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52968/0 (socket says 172.21.15.49:52968) 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 -- 172.21.15.49:0/957661349 learned_addr learned my addr 172.21.15.49:0/957661349 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.902 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f49237fe640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154f70 0x7f49240cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 -- 172.21.15.49:0/957661349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154f70 msgr2=0x7f49240cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154f70 0x7f49240cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 -- 172.21.15.49:0/957661349 >> v1:172.21.15.49:6789/0 conn(0x7f49240c6640 legacy=0x7f49240c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.902+0000 7f4923fff640 1 -- 172.21.15.49:0/957661349 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49240c5b60 con 0x7f4924154b90 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f4923fff640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154b90 0x7f49240cb280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4914009870 tx=0x7f491402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a9ec864129faf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f4922ffd640 1 -- 172.21.15.49:0/957661349 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914037070 con 0x7f4924154b90 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f4922ffd640 1 -- 172.21.15.49:0/957661349 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491402ae50 con 0x7f4924154b90 2024-05-09T08:58:39.903 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f4922ffd640 1 -- 172.21.15.49:0/957661349 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914032860 con 0x7f4924154b90 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 -- 172.21.15.49:0/957661349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154b90 msgr2=0x7f49240cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154b90 0x7f49240cb280 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4914009870 tx=0x7f491402a300 comp rx=0 tx=0).stop 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 -- 172.21.15.49:0/957661349 shutdown_connections 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154f70 0x7f49240cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 --2- 172.21.15.49:0/957661349 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154b90 0x7f49240cb280 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 -- 172.21.15.49:0/957661349 >> 172.21.15.49:0/957661349 conn(0x7f4924074840 msgr2=0x7f4924074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:39.906 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 -- 172.21.15.49:0/957661349 shutdown_connections 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.903+0000 7f492a7ce640 1 -- 172.21.15.49:0/957661349 wait complete. 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 Processor -- start 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 -- start start 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 0x7f492414c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 0x7f49241486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f492415af40 con 0x7f4924154f70 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49240c6120 con 0x7f49240c6640 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.905+0000 7f492a7ce640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4924071820 con 0x7f4924154b90 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52978/0 (socket says 172.21.15.49:52978) 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 -- 172.21.15.49:0/830750937 learned_addr learned my addr 172.21.15.49:0/830750937 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:39.907 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f49237fe640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 0x7f49241486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4923fff640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 0x7f492414c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 -- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 msgr2=0x7f49241486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 0x7f49241486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 -- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 msgr2=0x7f492414c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 0x7f492414c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 -- 172.21.15.49:0/830750937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49240cf9b0 con 0x7f4924154f70 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4923fff640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 0x7f492414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f49237fe640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 0x7f49241486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4928d6b640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c7a0 tx=0x7f491c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c003b70 con 0x7f4924154f70 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491c003d30 con 0x7f4924154f70 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492415a910 con 0x7f4924154f70 2024-05-09T08:58:39.908 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c00aa30 con 0x7f4924154f70 2024-05-09T08:58:39.909 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.906+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492414d0b0 con 0x7f4924154f70 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.907+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4924152650 con 0x7f4924154f70 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.908+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f491c00abd0 con 0x7f4924154f70 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.908+0000 7f4920ff9640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.908+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f491c08ada0 con 0x7f4924154f70 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.909+0000 7f4923fff640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:39.910 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.909+0000 7f4923fff640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 0x7f490406e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4914034000 tx=0x7f491402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:39.912 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:39.913+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4924152650 con 0x7f4924154f70 2024-05-09T08:58:40.095 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.096+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f492404f660 con 0x7f490406be30 2024-05-09T08:58:40.096 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.097+0000 7f4920ff9640 1 -- 172.21.15.49:0/830750937 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f492404f660 con 0x7f490406be30 2024-05-09T08:58:40.096 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 msgr2=0x7f490406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 0x7f490406e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4914034000 tx=0x7f491402e000 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 msgr2=0x7f49241f0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f491c00c7a0 tx=0x7f491c00cc60 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 shutdown_connections 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f490406be30 0x7f490406e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4924154f70 0x7f49241f0970 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4924154b90 0x7f49241486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 --2- 172.21.15.49:0/830750937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49240c6640 0x7f492414c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 >> 172.21.15.49:0/830750937 conn(0x7f4924074840 msgr2=0x7f492414a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 shutdown_connections 2024-05-09T08:58:40.099 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.099+0000 7f492a7ce640 1 -- 172.21.15.49:0/830750937 wait complete. 2024-05-09T08:58:40.109 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T08:58:40.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:40.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.220+0000 7f95798e7640 1 Processor -- start 2024-05-09T08:58:40.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.220+0000 7f95798e7640 1 -- start start 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95798e7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9574152b60 0x7f9574159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95798e7640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95740c78f0 0x7f95740c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95798e7640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95740c2550 con 0x7f95740c9ab0 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95798e7640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f95740c2b30 con 0x7f95740c78f0 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95798e7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f9574159b60 con 0x7f9574152b60 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95737fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95740c78f0 0x7f95740c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95737fe640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f95740c78f0 0x7f95740c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:52992/0 (socket says 172.21.15.49:52992) 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95737fe640 1 -- 172.21.15.49:0/4190005395 learned_addr learned my addr 172.21.15.49:0/4190005395 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f9572ffd640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9574152b60 0x7f9574159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.221+0000 7f95737fe640 1 -- 172.21.15.49:0/4190005395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9574152b60 msgr2=0x7f9574159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f95737fe640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9574152b60 0x7f9574159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f95737fe640 1 -- 172.21.15.49:0/4190005395 >> v1:172.21.15.183:6789/0 conn(0x7f95740c9ab0 legacy=0x7f95740c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.222 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f95737fe640 1 -- 172.21.15.49:0/4190005395 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95740c0820 con 0x7f95740c78f0 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f95737fe640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95740c78f0 0x7f95740c5a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f956c009e50 tx=0x7f956c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8073a4ec4f0c654 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f9571ffb640 1 -- 172.21.15.49:0/4190005395 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c036070 con 0x7f95740c78f0 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f9571ffb640 1 -- 172.21.15.49:0/4190005395 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c031600 con 0x7f95740c78f0 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.222+0000 7f9571ffb640 1 -- 172.21.15.49:0/4190005395 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c0319d0 con 0x7f95740c78f0 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 -- 172.21.15.49:0/4190005395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95740c78f0 msgr2=0x7f95740c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95740c78f0 0x7f95740c5a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f956c009e50 tx=0x7f956c02a750 comp rx=0 tx=0).stop 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 -- 172.21.15.49:0/4190005395 shutdown_connections 2024-05-09T08:58:40.223 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f95740c78f0 0x7f95740c5a10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 --2- 172.21.15.49:0/4190005395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9574152b60 0x7f9574159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 -- 172.21.15.49:0/4190005395 >> 172.21.15.49:0/4190005395 conn(0x7f95740c8e00 msgr2=0x7f95740c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 -- 172.21.15.49:0/4190005395 shutdown_connections 2024-05-09T08:58:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.223+0000 7f95798e7640 1 -- 172.21.15.49:0/4190005395 wait complete. 2024-05-09T08:58:40.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.224+0000 7f95798e7640 1 Processor -- start 2024-05-09T08:58:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.224+0000 7f95798e7640 1 -- start start 2024-05-09T08:58:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95740c78f0 0x7f95741f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9574152b60 0x7f95740bed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 0x7f95740bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f9574152f40 con 0x7f957414ed50 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95740c27c0 con 0x7f9574152b60 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95798e7640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f95741530c0 con 0x7f95740c78f0 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 0x7f95740bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95737fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 0x7f95740bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53008/0 (socket says 172.21.15.49:53008) 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95737fe640 1 -- 172.21.15.49:0/3098947510 learned_addr learned my addr 172.21.15.49:0/3098947510 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f9572ffd640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95740c78f0 0x7f95741f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.225+0000 7f95727fc640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9574152b60 0x7f95740bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 -- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95740c78f0 msgr2=0x7f95741f0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95740c78f0 0x7f95741f0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 -- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9574152b60 msgr2=0x7f95740bed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9574152b60 0x7f95740bed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741f0e40 con 0x7f957414ed50 2024-05-09T08:58:40.228 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95727fc640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9574152b60 0x7f95740bed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95737fe640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 0x7f95740bf2c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f956c034040 tx=0x7f956c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c036050 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c042070 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741f10d0 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c009510 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.226+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95741f1690 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.227+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9574068560 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.227+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f956c007bf0 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f956bfff640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f955406be60 0x7f955406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f956c0abe10 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f956bfff640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f95540742c0 0x7f9554076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f956c0020b0 con 0x7f95540742c0 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f956c0ae510 con 0x7f957414ed50 2024-05-09T08:58:40.229 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f9572ffd640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f955406be60 0x7f955406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.228+0000 7f95727fc640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f95540742c0 0x7f9554076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.229+0000 7f9572ffd640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f955406be60 0x7f955406e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f956400aa20 tx=0x7f9564008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.229+0000 7f95727fc640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f95540742c0 0x7f9554076760 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.230 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.229+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== osd.4 v2:172.21.15.183:6808/173400960 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f956c0020b0 con 0x7f95540742c0 2024-05-09T08:58:40.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.243+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 --> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f95740c7390 con 0x7f95540742c0 2024-05-09T08:58:40.242 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.243+0000 7f956bfff640 1 -- 172.21.15.49:0/3098947510 <== osd.4 v2:172.21.15.183:6808/173400960 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f95740c7390 con 0x7f95540742c0 2024-05-09T08:58:40.243 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:40.243 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:40.243 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f95540742c0 msgr2=0x7f9554076760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6808/173400960,v1:172.21.15.183:6809/173400960] conn(0x7f95540742c0 0x7f9554076760 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f955406be60 msgr2=0x7f955406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f955406be60 0x7f955406e320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f956400aa20 tx=0x7f9564008040 comp rx=0 tx=0).stop 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 msgr2=0x7f95740bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 --2- 172.21.15.49:0/3098947510 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f957414ed50 0x7f95740bf2c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f956c034040 tx=0x7f956c004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95737fe640 1 -- 172.21.15.49:0/3098947510 reap_dead start 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 shutdown_connections 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 >> 172.21.15.49:0/3098947510 conn(0x7f95740c8e00 msgr2=0x7f95740cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 shutdown_connections 2024-05-09T08:58:40.244 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.244+0000 7f95798e7640 1 -- 172.21.15.49:0/3098947510 wait complete. 2024-05-09T08:58:40.254 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T08:58:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.359+0000 7f743c97e640 1 Processor -- start 2024-05-09T08:58:40.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.359+0000 7f743c97e640 1 -- start start 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743c97e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7438155180 0x7f74380c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743c97e640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74380c2730 0x7f74380c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743c97e640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f743815f160 con 0x7f74380c8f90 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743c97e640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f74380c7b90 con 0x7f74380c2730 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743c97e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74380c78b0 con 0x7f7438155180 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74380c2730 0x7f74380c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f74380c2730 0x7f74380c2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53014/0 (socket says 172.21.15.49:53014) 2024-05-09T08:58:40.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 -- 172.21.15.49:0/838793041 learned_addr learned my addr 172.21.15.49:0/838793041 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f743659c640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7438155180 0x7f74380c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 -- 172.21.15.49:0/838793041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7438155180 msgr2=0x7f74380c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7438155180 0x7f74380c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 -- 172.21.15.49:0/838793041 >> v1:172.21.15.183:6789/0 conn(0x7f74380c8f90 legacy=0x7f743806b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.360+0000 7f7436d9d640 1 -- 172.21.15.49:0/838793041 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743815e3a0 con 0x7f74380c2730 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.361+0000 7f7436d9d640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74380c2bc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e50 tx=0x7f743002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9ad0ba9b92d8c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.361+0000 7f7435d9b640 1 -- 172.21.15.49:0/838793041 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430036070 con 0x7f74380c2730 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.361+0000 7f7435d9b640 1 -- 172.21.15.49:0/838793041 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7430031600 con 0x7f74380c2730 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.361+0000 7f7435d9b640 1 -- 172.21.15.49:0/838793041 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300319d0 con 0x7f74380c2730 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 -- 172.21.15.49:0/838793041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 msgr2=0x7f74380c2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74380c2bc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7430009e50 tx=0x7f743002a680 comp rx=0 tx=0).stop 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 -- 172.21.15.49:0/838793041 shutdown_connections 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74380c2bc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 --2- 172.21.15.49:0/838793041 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7438155180 0x7f74380c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 -- 172.21.15.49:0/838793041 >> 172.21.15.49:0/838793041 conn(0x7f74380c9650 msgr2=0x7f74380c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 -- 172.21.15.49:0/838793041 shutdown_connections 2024-05-09T08:58:40.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.362+0000 7f743c97e640 1 -- 172.21.15.49:0/838793041 wait complete. 2024-05-09T08:58:40.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.363+0000 7f743c97e640 1 Processor -- start 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.363+0000 7f743c97e640 1 -- start start 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74380c8f90 0x7f743814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 0x7f7438148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f743806b910 con 0x7f74380c2730 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f743815e960 con 0x7f7438155180 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743c97e640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f74380c7550 con 0x7f74380c8f90 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53018/0 (socket says 172.21.15.49:53018) 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 -- 172.21.15.49:0/2230381034 learned_addr learned my addr 172.21.15.49:0/2230381034 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f742ffff640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74380c8f90 0x7f743814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f7436d9d640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 0x7f7438148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 -- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74380c8f90 msgr2=0x7f743814acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74380c8f90 0x7f743814acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 -- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 msgr2=0x7f7438148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 0x7f7438148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f743659c640 1 -- 172.21.15.49:0/2230381034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74381466e0 con 0x7f74380c2730 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.364+0000 7f7436d9d640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 0x7f7438148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f743659c640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f742800d5d0 tx=0x7f742800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74280125f0 con 0x7f74380c2730 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7428019070 con 0x7f74380c2730 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74280129f0 con 0x7f74380c2730 2024-05-09T08:58:40.369 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743815dfe0 con 0x7f74380c2730 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.365+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743815d500 con 0x7f74380c2730 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.366+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743804f660 con 0x7f74380c2730 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.366+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7428012b90 con 0x7f74380c2730 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.367+0000 7f742effd640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 0x7f741806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.367+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f742808b340 con 0x7f74380c2730 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.367+0000 7f742ffff640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 0x7f741806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.368+0000 7f742ffff640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 0x7f741806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7438153340 tx=0x7f7420009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.370 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.371+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7428059c40 con 0x7f74380c2730 2024-05-09T08:58:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.554+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f7438146c80 con 0x7f741806be30 2024-05-09T08:58:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.555+0000 7f742effd640 1 -- 172.21.15.49:0/2230381034 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7438146c80 con 0x7f741806be30 2024-05-09T08:58:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T08:58:40.556 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 msgr2=0x7f741806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 0x7f741806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7438153340 tx=0x7f7420009040 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 msgr2=0x7f74381478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f742800d5d0 tx=0x7f742800da90 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 shutdown_connections 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f741806be30 0x7f741806e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7438155180 0x7f7438148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74380c8f90 0x7f743814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 --2- 172.21.15.49:0/2230381034 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f74380c2730 0x7f74381478d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 >> 172.21.15.49:0/2230381034 conn(0x7f74380c9650 msgr2=0x7f7438153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 shutdown_connections 2024-05-09T08:58:40.557 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.557+0000 7f743c97e640 1 -- 172.21.15.49:0/2230381034 wait complete. 2024-05-09T08:58:40.567 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T08:58:40.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:40.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.676+0000 7f0ea8438640 1 Processor -- start 2024-05-09T08:58:40.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.677+0000 7f0ea8438640 1 -- start start 2024-05-09T08:58:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.677+0000 7f0ea8438640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0ea006e3f0 0x7f0ea00c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.677+0000 7f0ea8438640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ea0067bd0 0x7f0ea015b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.677+0000 7f0ea8438640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea00c7990 con 0x7f0ea006e7d0 2024-05-09T08:58:40.677 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea8438640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea015b400 con 0x7f0ea0067bd0 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea8438640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea015add0 con 0x7f0ea006e3f0 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ea0067bd0 0x7f0ea015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f0ea0067bd0 0x7f0ea015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53030/0 (socket says 172.21.15.49:53030) 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 -- 172.21.15.49:0/438900497 learned_addr learned my addr 172.21.15.49:0/438900497 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea61d4640 1 --2- 172.21.15.49:0/438900497 >> v2:172.21.15.183:3300/0 conn(0x7f0ea006e3f0 0x7f0ea00c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 -- 172.21.15.49:0/438900497 >> v2:172.21.15.183:3300/0 conn(0x7f0ea006e3f0 msgr2=0x7f0ea00c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/438900497 >> v2:172.21.15.183:3300/0 conn(0x7f0ea006e3f0 0x7f0ea00c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 -- 172.21.15.49:0/438900497 >> v1:172.21.15.49:6789/0 conn(0x7f0ea006e7d0 legacy=0x7f0ea0154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.678 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.678+0000 7f0ea59d3640 1 -- 172.21.15.49:0/438900497 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea00c2be0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/438900497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea015b890 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00c9860 tx=0x7f0e9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53fbc0aa1a5e0be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea51d2640 1 -- 172.21.15.49:0/438900497 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90037070 con 0x7f0ea0067bd0 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea51d2640 1 -- 172.21.15.49:0/438900497 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e90032440 con 0x7f0ea0067bd0 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea51d2640 1 -- 172.21.15.49:0/438900497 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90032810 con 0x7f0ea0067bd0 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea8438640 1 -- 172.21.15.49:0/438900497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 msgr2=0x7f0ea015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.679+0000 7f0ea8438640 1 --2- 172.21.15.49:0/438900497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea015b890 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00c9860 tx=0x7f0e9002a270 comp rx=0 tx=0).stop 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 -- 172.21.15.49:0/438900497 shutdown_connections 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 --2- 172.21.15.49:0/438900497 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea015b890 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 --2- 172.21.15.49:0/438900497 >> v2:172.21.15.183:3300/0 conn(0x7f0ea006e3f0 0x7f0ea00c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 -- 172.21.15.49:0/438900497 >> 172.21.15.49:0/438900497 conn(0x7f0ea00cce00 msgr2=0x7f0ea00cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 -- 172.21.15.49:0/438900497 shutdown_connections 2024-05-09T08:58:40.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.680+0000 7f0ea8438640 1 -- 172.21.15.49:0/438900497 wait complete. 2024-05-09T08:58:40.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 Processor -- start 2024-05-09T08:58:40.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 -- start start 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea0153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ea006e3f0 0x7f0ea01f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ea006e7d0 0x7f0ea01f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea015b090 con 0x7f0ea0067bd0 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea00c31a0 con 0x7f0ea006e3f0 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.681+0000 7f0ea8438640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea00c2830 con 0x7f0ea006e7d0 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea0153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea0153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53038/0 (socket says 172.21.15.49:53038) 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 -- 172.21.15.49:0/2486803161 learned_addr learned my addr 172.21.15.49:0/2486803161 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ea006e3f0 0x7f0ea01f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea69d5640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ea006e7d0 0x7f0ea01f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 -- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ea006e7d0 msgr2=0x7f0ea01f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ea006e7d0 0x7f0ea01f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 -- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ea006e3f0 msgr2=0x7f0ea01f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ea006e3f0 0x7f0ea01f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea01f3be0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea69d5640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ea006e7d0 0x7f0ea01f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ea006e3f0 0x7f0ea01f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea61d4640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea0153dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9e0 tx=0x7f0e9c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0ea0067bd0 2024-05-09T08:58:40.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00c9d0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00cdc0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea0156ce0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.682+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea01f4140 con 0x7f0ea0067bd0 2024-05-09T08:58:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.683+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ea006c090 con 0x7f0ea0067bd0 2024-05-09T08:58:40.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.684+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e9c01a070 con 0x7f0ea0067bd0 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.684+0000 7f0e96ffd640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0e8406be60 0x7f0e8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.684+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f0e9c08b330 con 0x7f0ea0067bd0 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0e96ffd640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f0e840742c0 0x7f0e84076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e84076e30 con 0x7f0e840742c0 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0e9c08b7c0 con 0x7f0ea0067bd0 2024-05-09T08:58:40.686 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0e8406be60 0x7f0e8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0ea69d5640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f0e840742c0 0x7f0e84076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0ea59d3640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0e8406be60 0x7f0e8406e320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034000 tx=0x7f0e9002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.685+0000 7f0ea69d5640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f0e840742c0 0x7f0e84076760 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.687 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.686+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== osd.5 v2:172.21.15.183:6800/4279483611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e84076e30 con 0x7f0e840742c0 2024-05-09T08:58:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.700+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 --> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0ea01f4ad0 con 0x7f0e840742c0 2024-05-09T08:58:40.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.700+0000 7f0e96ffd640 1 -- 172.21.15.49:0/2486803161 <== osd.5 v2:172.21.15.183:6800/4279483611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0ea01f4ad0 con 0x7f0e840742c0 2024-05-09T08:58:40.700 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:40.700 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:40.700 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.700+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f0e840742c0 msgr2=0x7f0e84076760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.700+0000 7f0ea8438640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6800/4279483611,v1:172.21.15.183:6801/4279483611] conn(0x7f0e840742c0 0x7f0e84076760 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0e8406be60 msgr2=0x7f0e8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f0e8406be60 0x7f0e8406e320 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e90034000 tx=0x7f0e9002e000 comp rx=0 tx=0).stop 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 msgr2=0x7f0ea0153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 --2- 172.21.15.49:0/2486803161 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f0ea0067bd0 0x7f0ea0153dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00e9e0 tx=0x7f0e9c00eea0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea69d5640 1 -- 172.21.15.49:0/2486803161 reap_dead start 2024-05-09T08:58:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 shutdown_connections 2024-05-09T08:58:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 >> 172.21.15.49:0/2486803161 conn(0x7f0ea00cce00 msgr2=0x7f0ea015a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 shutdown_connections 2024-05-09T08:58:40.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.701+0000 7f0ea8438640 1 -- 172.21.15.49:0/2486803161 wait complete. 2024-05-09T08:58:40.710 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T08:58:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.816+0000 7fe7f6cdd640 1 Processor -- start 2024-05-09T08:58:40.815 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.816+0000 7fe7f6cdd640 1 -- start start 2024-05-09T08:58:40.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f6cdd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe7f0152e30 0x7fe7f0158be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f6cdd640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f0159b10 con 0x7fe7f0072670 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f6cdd640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f0159270 con 0x7fe7f00c7130 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f6cdd640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f0159550 con 0x7fe7f0152e30 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe7f0152e30 0x7fe7f0158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fe7f0152e30 0x7fe7f0158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53046/0 (socket says 172.21.15.49:53046) 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 -- 172.21.15.49:0/846642142 learned_addr learned my addr 172.21.15.49:0/846642142 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 -- 172.21.15.49:0/846642142 >> v1:172.21.15.183:6789/0 conn(0x7fe7f00c7130 legacy=0x7fe7f00c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 -- 172.21.15.49:0/846642142 >> v1:172.21.15.49:6789/0 conn(0x7fe7f0072670 legacy=0x7fe7f00c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.817+0000 7fe7f4a79640 1 -- 172.21.15.49:0/846642142 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0159830 con 0x7fe7f0152e30 2024-05-09T08:58:40.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.818+0000 7fe7f4a79640 1 --2- 172.21.15.49:0/846642142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f0158be0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009870 tx=0x7fe7e802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4308321492386bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.818+0000 7fe7ef7fe640 1 -- 172.21.15.49:0/846642142 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8037070 con 0x7fe7f0152e30 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.818+0000 7fe7ef7fe640 1 -- 172.21.15.49:0/846642142 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e8032440 con 0x7fe7f0152e30 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.818+0000 7fe7ef7fe640 1 -- 172.21.15.49:0/846642142 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8032810 con 0x7fe7f0152e30 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/846642142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 msgr2=0x7fe7f0158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/846642142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f0158be0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009870 tx=0x7fe7e802a2a0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/846642142 shutdown_connections 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/846642142 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f0158be0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/846642142 >> 172.21.15.49:0/846642142 conn(0x7fe7f00cc290 msgr2=0x7fe7f00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/846642142 shutdown_connections 2024-05-09T08:58:40.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.819+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/846642142 wait complete. 2024-05-09T08:58:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.820+0000 7fe7f6cdd640 1 Processor -- start 2024-05-09T08:58:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.820+0000 7fe7f6cdd640 1 -- start start 2024-05-09T08:58:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 0x7fe7f00c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 0x7fe7f00c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.821 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00c6620 con 0x7fe7f0152e30 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00c59f0 con 0x7fe7f0072670 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f6cdd640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f006b4b0 con 0x7fe7f00c7130 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53056/0 (socket says 172.21.15.49:53056) 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 -- 172.21.15.49:0/2101511249 learned_addr learned my addr 172.21.15.49:0/2101511249 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7effff640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 0x7fe7f00c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f4a79640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 0x7fe7f00c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 -- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 msgr2=0x7fe7f00c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 0x7fe7f00c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 -- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 msgr2=0x7fe7f00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:40.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 0x7fe7f00c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 -- 172.21.15.49:0/2101511249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0156550 con 0x7fe7f0152e30 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7effff640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 0x7fe7f00c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f4a79640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 0x7fe7f00c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.821+0000 7fe7f527a640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c3c0 tx=0x7fe7d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.822+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8017070 con 0x7fe7f0152e30 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.822+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d80052d0 con 0x7fe7f0152e30 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.822+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01e4fd0 con 0x7fe7f0152e30 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.822+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8003810 con 0x7fe7f0152e30 2024-05-09T08:58:40.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.822+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01e5590 con 0x7fe7f0152e30 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.823+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f006d1d0 con 0x7fe7f0152e30 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.823+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7d801a070 con 0x7fe7f0152e30 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.824+0000 7fe7ed7fa640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 0x7fe7d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.824+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fe7d808a1f0 con 0x7fe7f0152e30 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.824+0000 7fe7f4a79640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 0x7fe7d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:40.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.824+0000 7fe7f4a79640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 0x7fe7d006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8034000 tx=0x7fe7e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:40.827 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:40.828+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7f006d1d0 con 0x7fe7f0152e30 2024-05-09T08:58:41.008 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:58:41.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.011+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe7f00cbb70 con 0x7fe7d006be30 2024-05-09T08:58:41.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:58:41.012 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.011+0000 7fe7ed7fa640 1 -- 172.21.15.49:0/2101511249 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe7f00cbb70 con 0x7fe7d006be30 2024-05-09T08:58:41.012 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T08:58:41.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.013+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 msgr2=0x7fe7d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.013+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 0x7fe7d006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8034000 tx=0x7fe7e802e000 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.013+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 msgr2=0x7fe7f01e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.013+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe7d800c3c0 tx=0x7fe7d800c880 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 shutdown_connections 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7d006be30 0x7fe7d006e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7f0152e30 0x7fe7f01e2360 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7f00c7130 0x7fe7f00c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 --2- 172.21.15.49:0/2101511249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f0072670 0x7fe7f00c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 >> 172.21.15.49:0/2101511249 conn(0x7fe7f00cc290 msgr2=0x7fe7f0156790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 shutdown_connections 2024-05-09T08:58:41.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.014+0000 7fe7f6cdd640 1 -- 172.21.15.49:0/2101511249 wait complete. 2024-05-09T08:58:41.024 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T08:58:41.025 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.214+0000 7f8685cb3640 1 Processor -- start 2024-05-09T08:58:41.213 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.214+0000 7f8685cb3640 1 -- start start 2024-05-09T08:58:41.217 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f8685cb3640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f86801520c0 0x7f868006c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f8685cb3640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868006b600 con 0x7f8680151ce0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f8685cb3640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868006b8a0 con 0x7f86800c4dc0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f8685cb3640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f868006bb80 con 0x7f86801520c0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f867f7fe640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f8680151ce0 0x7f8680158670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49782/0 (socket says 172.21.15.49:49782) 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f867f7fe640 1 -- 172.21.15.49:0/2468839784 learned_addr learned my addr 172.21.15.49:0/2468839784 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511562819 0 0) 0x7f868006b600 con 0x7f8680151ce0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f867effd640 1 --2- 172.21.15.49:0/2468839784 >> v2:172.21.15.183:3300/0 conn(0x7f86801520c0 0x7f868006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.215+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8668005730 con 0x7f8680151ce0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2635141510 0 0) 0x7f8668005730 con 0x7f8680151ce0 2024-05-09T08:58:41.218 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 >> v2:172.21.15.183:3300/0 conn(0x7f86801520c0 msgr2=0x7f868006c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 --2- 172.21.15.49:0/2468839784 >> v2:172.21.15.183:3300/0 conn(0x7f86801520c0 0x7f868006c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 >> v1:172.21.15.183:6789/0 conn(0x7f86800c4dc0 legacy=0x7f86800c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868006be60 con 0x7f8680151ce0 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f8670001670 con 0x7f8680151ce0 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 >> v1:172.21.15.49:6789/0 conn(0x7f8680151ce0 legacy=0x7f8680158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8668005bc0 0x7f8668005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8668006a10 0x7f8668006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f866800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f868006bb80 con 0x7f86680073d0 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f866800e270 con 0x7f8668006a10 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f866800e470 con 0x7f8668005bc0 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f866800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.219 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867effd640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8668006a10 0x7f8668006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867f7fe640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8668005bc0 0x7f8668005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 -- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8668005bc0 msgr2=0x7f8668005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8668005bc0 0x7f8668005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 -- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8668006a10 msgr2=0x7f8668006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8668006a10 0x7f8668006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.216+0000 7f867ffff640 1 -- 172.21.15.49:0/2468839784 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8668005730 con 0x7f86680073d0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f867ffff640 1 -- 172.21.15.49:0/2468839784 reap_dead start 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f867ffff640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f866800dbd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8678001e50 tx=0x7f8668007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98550af98d690d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678001a70 con 0x7f86680073d0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8678003970 con 0x7f86680073d0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f867e7fc640 1 -- 172.21.15.49:0/2468839784 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678003d60 con 0x7f86680073d0 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f8685cb3640 1 -- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 msgr2=0x7f866800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.220 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.217+0000 7f8685cb3640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f866800dbd0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8678001e50 tx=0x7f8668007a40 comp rx=0 tx=0).stop 2024-05-09T08:58:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.218+0000 7f8685cb3640 1 -- 172.21.15.49:0/2468839784 shutdown_connections 2024-05-09T08:58:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.218+0000 7f8685cb3640 1 --2- 172.21.15.49:0/2468839784 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f866800dbd0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.218+0000 7f8685cb3640 1 -- 172.21.15.49:0/2468839784 >> 172.21.15.49:0/2468839784 conn(0x7f86800c8e00 msgr2=0x7f86800c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.218+0000 7f8685cb3640 1 -- 172.21.15.49:0/2468839784 shutdown_connections 2024-05-09T08:58:41.221 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.218+0000 7f8685cb3640 1 -- 172.21.15.49:0/2468839784 wait complete. 2024-05-09T08:58:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.219+0000 7f8685cb3640 1 Processor -- start 2024-05-09T08:58:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.219+0000 7f8685cb3640 1 -- start start 2024-05-09T08:58:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f86801542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680155140 0x7f86801e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.224 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86801e2f50 0x7f8680143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f86800c51d0 con 0x7f86680073d0 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f868006c420 con 0x7f8680155140 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f8685cb3640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f868006c940 con 0x7f86801e2f50 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f86801542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f86801542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53076/0 (socket says 172.21.15.49:53076) 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 -- 172.21.15.49:0/269676051 learned_addr learned my addr 172.21.15.49:0/269676051 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867ffff640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86801e2f50 0x7f8680143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867effd640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680155140 0x7f86801e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 -- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86801e2f50 msgr2=0x7f8680143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f86801e2f50 0x7f8680143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 -- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680155140 msgr2=0x7f86801e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680155140 0x7f86801e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.225 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.220+0000 7f867f7fe640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86801e3b00 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f867f7fe640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f86801542b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f86700072c0 tx=0x7f86700076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867000f8e0 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8670018070 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867000fce0 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86801e3540 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.221+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801e2210 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.222+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8680068560 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f867001d460 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f866c06be30 0x7f866c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f867008aed0 con 0x7f86680073d0 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f866c074290 0x7f866c076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8670015e70 con 0x7f866c074290 2024-05-09T08:58:41.226 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f867effd640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f866c06be30 0x7f866c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f867008d5d0 con 0x7f86680073d0 2024-05-09T08:58:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.223+0000 7f867ffff640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f866c074290 0x7f866c076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.224+0000 7f867effd640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f866c06be30 0x7f866c06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f868014bd50 tx=0x7f8674009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.224+0000 7f867ffff640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f866c074290 0x7f866c076730 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.227 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.225+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== osd.6 v2:172.21.15.183:6824/3521264583 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8670015e70 con 0x7f866c074290 2024-05-09T08:58:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.238+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 --> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f86800c5930 con 0x7f866c074290 2024-05-09T08:58:41.237 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.238+0000 7f8667fff640 1 -- 172.21.15.49:0/269676051 <== osd.6 v2:172.21.15.183:6824/3521264583 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f86800c5930 con 0x7f866c074290 2024-05-09T08:58:41.238 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:41.238 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:41.238 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f866c074290 msgr2=0x7f866c076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6824/3521264583,v1:172.21.15.183:6825/3521264583] conn(0x7f866c074290 0x7f866c076730 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f866c06be30 msgr2=0x7f866c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f866c06be30 0x7f866c06e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f868014bd50 tx=0x7f8674009190 comp rx=0 tx=0).stop 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 msgr2=0x7f86801542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 --2- 172.21.15.49:0/269676051 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f86680073d0 0x7f86801542b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f86700072c0 tx=0x7f86700076c0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f867ffff640 1 -- 172.21.15.49:0/269676051 reap_dead start 2024-05-09T08:58:41.239 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 shutdown_connections 2024-05-09T08:58:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 >> 172.21.15.49:0/269676051 conn(0x7f86800c8e00 msgr2=0x7f86800c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 shutdown_connections 2024-05-09T08:58:41.240 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.239+0000 7f8685cb3640 1 -- 172.21.15.49:0/269676051 wait complete. 2024-05-09T08:58:41.249 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T08:58:41.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.355+0000 7f2b0d64d640 1 Processor -- start 2024-05-09T08:58:41.354 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.355+0000 7f2b0d64d640 1 -- start start 2024-05-09T08:58:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b0d64d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b08151be0 0x7f2b080c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b0d64d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b080c73a0 0x7f2b0806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b0d64d640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b080c4b10 con 0x7f2b0815af60 2024-05-09T08:58:41.356 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b0d64d640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f2b080c5c70 con 0x7f2b08151be0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b0d64d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2b080c4da0 con 0x7f2b080c73a0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b08151be0 0x7f2b080c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f2b08151be0 0x7f2b080c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53084/0 (socket says 172.21.15.49:53084) 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 -- 172.21.15.49:0/3517789337 learned_addr learned my addr 172.21.15.49:0/3517789337 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b077fe640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b080c73a0 0x7f2b0806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 -- 172.21.15.49:0/3517789337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b080c73a0 msgr2=0x7f2b0806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b080c73a0 0x7f2b0806dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 -- 172.21.15.49:0/3517789337 >> v1:172.21.15.183:6789/0 conn(0x7f2b0815af60 legacy=0x7f2b080c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.356+0000 7f2b06ffd640 1 -- 172.21.15.49:0/3517789337 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b080c5060 con 0x7f2b08151be0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.357+0000 7f2b06ffd640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151be0 0x7f2b080c45d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009870 tx=0x7f2af802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48112beb9e9ee8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.357 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.357+0000 7f2b067fc640 1 -- 172.21.15.49:0/3517789337 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8037070 con 0x7f2b08151be0 2024-05-09T08:58:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.357+0000 7f2b067fc640 1 -- 172.21.15.49:0/3517789337 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af802ae50 con 0x7f2b08151be0 2024-05-09T08:58:41.358 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.357+0000 7f2b067fc640 1 -- 172.21.15.49:0/3517789337 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8032860 con 0x7f2b08151be0 2024-05-09T08:58:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 -- 172.21.15.49:0/3517789337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151be0 msgr2=0x7f2b080c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.360 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151be0 0x7f2b080c45d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009870 tx=0x7f2af802a300 comp rx=0 tx=0).stop 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 -- 172.21.15.49:0/3517789337 shutdown_connections 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b080c73a0 0x7f2b0806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/3517789337 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b08151be0 0x7f2b080c45d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 -- 172.21.15.49:0/3517789337 >> 172.21.15.49:0/3517789337 conn(0x7f2b080cc290 msgr2=0x7f2b080cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 -- 172.21.15.49:0/3517789337 shutdown_connections 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.358+0000 7f2b0d64d640 1 -- 172.21.15.49:0/3517789337 wait complete. 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.359+0000 7f2b0d64d640 1 Processor -- start 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.359+0000 7f2b0d64d640 1 -- start start 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b08151be0 0x7f2b08158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b081eeda0 0x7f2b08154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2b080c5900 con 0x7f2b0815af60 2024-05-09T08:58:41.361 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b080c5620 con 0x7f2b08151be0 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2b08151fc0 con 0x7f2b081eeda0 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53088/0 (socket says 172.21.15.49:53088) 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 -- 172.21.15.49:0/4194603123 learned_addr learned my addr 172.21.15.49:0/4194603123 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b077fe640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b081eeda0 0x7f2b08154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b06ffd640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b08151be0 0x7f2b08158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 -- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b081eeda0 msgr2=0x7f2b08154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b081eeda0 0x7f2b08154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 -- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b08151be0 msgr2=0x7f2b08158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b08151be0 0x7f2b08158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 -- 172.21.15.49:0/4194603123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b081ef2c0 con 0x7f2b0815af60 2024-05-09T08:58:41.362 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afe7fc640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c480 tx=0x7f2af000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af00052d0 con 0x7f2b0815af60 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0005470 con 0x7f2b0815af60 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b08155af0 con 0x7f2b0815af60 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0012870 con 0x7f2b0815af60 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.360+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b081560b0 con 0x7f2b0815af60 2024-05-09T08:58:41.363 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.361+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0804f660 con 0x7f2b0815af60 2024-05-09T08:58:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.362+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2af0012a10 con 0x7f2b0815af60 2024-05-09T08:58:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.362+0000 7f2afffff640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 0x7f2ae806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.362+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f2af008aa20 con 0x7f2b0815af60 2024-05-09T08:58:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.363+0000 7f2b06ffd640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 0x7f2ae806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.364 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.363+0000 7f2b06ffd640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 0x7f2ae806e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2af8034000 tx=0x7f2af802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.366 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.367+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2af0058d60 con 0x7f2b0815af60 2024-05-09T08:58:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.551+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f2b08150e20 con 0x7f2ae806be30 2024-05-09T08:58:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.552+0000 7f2afffff640 1 -- 172.21.15.49:0/4194603123 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b08150e20 con 0x7f2ae806be30 2024-05-09T08:58:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T08:58:41.553 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 msgr2=0x7f2ae806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 0x7f2ae806e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2af8034000 tx=0x7f2af802e000 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 msgr2=0x7f2b08153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2af000c480 tx=0x7f2af000c940 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 shutdown_connections 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f2ae806be30 0x7f2ae806e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b081eeda0 0x7f2b08154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2b0815af60 0x7f2b08153380 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 --2- 172.21.15.49:0/4194603123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b08151be0 0x7f2b08158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 >> 172.21.15.49:0/4194603123 conn(0x7f2b080cc290 msgr2=0x7f2b080722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 shutdown_connections 2024-05-09T08:58:41.554 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.554+0000 7f2b0d64d640 1 -- 172.21.15.49:0/4194603123 wait complete. 2024-05-09T08:58:41.564 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T08:58:41.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T08:58:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.673+0000 7fefd7509640 1 Processor -- start 2024-05-09T08:58:41.672 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.673+0000 7fefd7509640 1 -- start start 2024-05-09T08:58:41.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 0x7fefd0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c9ab0 0x7fefd00c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefd00c78f0 0x7fefd00c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fefd00c2b30 con 0x7fefd00c78f0 2024-05-09T08:58:41.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fefd00c2550 con 0x7fefd0152b60 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd7509640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefd0159b60 con 0x7fefd00c9ab0 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefd00c78f0 0x7fefd00c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fefd00c78f0 0x7fefd00c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53100/0 (socket says 172.21.15.49:53100) 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1316428996 learned_addr learned my addr 172.21.15.49:0/1316428996 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd52a5640 1 --2- 172.21.15.49:0/1316428996 >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 0x7fefd0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd4aa4640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c9ab0 0x7fefd00c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1316428996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c9ab0 msgr2=0x7fefd00c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c9ab0 0x7fefd00c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1316428996 >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 msgr2=0x7fefd0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1316428996 >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 0x7fefd0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1316428996 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd00c3500 con 0x7fefd00c78f0 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.674+0000 7fefd52a5640 1 --2- 172.21.15.49:0/1316428996 >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 0x7fefd0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:41.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.675+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd00c78f0 0x7fefd00c5a10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefd006f920 tx=0x7fefc802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1074fa789dfc62dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.675+0000 7fefc7fff640 1 -- 172.21.15.49:0/1316428996 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8036070 con 0x7fefd00c78f0 2024-05-09T08:58:41.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.675+0000 7fefc7fff640 1 -- 172.21.15.49:0/1316428996 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8031600 con 0x7fefd00c78f0 2024-05-09T08:58:41.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.675+0000 7fefc7fff640 1 -- 172.21.15.49:0/1316428996 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc80319d0 con 0x7fefd00c78f0 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 -- 172.21.15.49:0/1316428996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd00c78f0 msgr2=0x7fefd00c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd00c78f0 0x7fefd00c5a10 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefd006f920 tx=0x7fefc802a730 comp rx=0 tx=0).stop 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 -- 172.21.15.49:0/1316428996 shutdown_connections 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd00c78f0 0x7fefd00c5a10 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 --2- 172.21.15.49:0/1316428996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c9ab0 0x7fefd00c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 --2- 172.21.15.49:0/1316428996 >> v2:172.21.15.183:3300/0 conn(0x7fefd0152b60 0x7fefd0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.679 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 -- 172.21.15.49:0/1316428996 >> 172.21.15.49:0/1316428996 conn(0x7fefd00c8e00 msgr2=0x7fefd00c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 -- 172.21.15.49:0/1316428996 shutdown_connections 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.676+0000 7fefd7509640 1 -- 172.21.15.49:0/1316428996 wait complete. 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.677+0000 7fefd7509640 1 Processor -- start 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.677+0000 7fefd7509640 1 -- start start 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c78f0 0x7fefd015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefd0152b60 0x7fefd0158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 0x7fefd015c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00c27c0 con 0x7fefd014c630 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefd00c0b00 con 0x7fefd0152b60 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fefd0152f40 con 0x7fefd00c78f0 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 0x7fefd015c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 0x7fefd015c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53112/0 (socket says 172.21.15.49:53112) 2024-05-09T08:58:41.680 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1965867379 learned_addr learned my addr 172.21.15.49:0/1965867379 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd52a5640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c78f0 0x7fefd015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd4aa4640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefd0152b60 0x7fefd0158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c78f0 msgr2=0x7fefd015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefd00c78f0 0x7fefd015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefd0152b60 msgr2=0x7fefd0158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefd0152b60 0x7fefd0158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd01f1f40 con 0x7fefd014c630 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd4aa4640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefd0152b60 0x7fefd0158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd5aa6640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 0x7fefd015c450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fefc8034040 tx=0x7fefc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8036050 con 0x7fefd014c630 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefc8042070 con 0x7fefd014c630 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd01f2220 con 0x7fefd014c630 2024-05-09T08:58:41.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.678+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8009510 con 0x7fefd014c630 2024-05-09T08:58:41.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.679+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01f27e0 con 0x7fefd014c630 2024-05-09T08:58:41.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.679+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefd0068560 con 0x7fefd014c630 2024-05-09T08:58:41.682 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.680+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefc8007bf0 con 0x7fefd014c630 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.680+0000 7fefc5ffb640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fefb406be30 0x7fefb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fefc80abc80 con 0x7fefd014c630 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefc5ffb640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fefb4074290 0x7fefb4076730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefd52a5640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fefb406be30 0x7fefb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefb4076e00 con 0x7fefb4074290 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fefc80ae380 con 0x7fefd014c630 2024-05-09T08:58:41.683 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefd4aa4640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fefb4074290 0x7fefb4076730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefd52a5640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fefb406be30 0x7fefb406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fefcc006fd0 tx=0x7fefcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.681+0000 7fefd4aa4640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fefb4074290 0x7fefb4076730 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.684 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.682+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== osd.7 v2:172.21.15.183:6816/2743896476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefb4076e00 con 0x7fefb4074290 2024-05-09T08:58:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 --> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fefd015c070 con 0x7fefb4074290 2024-05-09T08:58:41.695 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefc5ffb640 1 -- 172.21.15.49:0/1965867379 <== osd.7 v2:172.21.15.183:6816/2743896476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fefd015c070 con 0x7fefb4074290 2024-05-09T08:58:41.695 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-05-09T08:58:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T08:58:41.695 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fefb4074290 msgr2=0x7fefb4076730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6816/2743896476,v1:172.21.15.183:6817/2743896476] conn(0x7fefb4074290 0x7fefb4076730 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fefb406be30 msgr2=0x7fefb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fefb406be30 0x7fefb406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fefcc006fd0 tx=0x7fefcc008040 comp rx=0 tx=0).stop 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 msgr2=0x7fefd015c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd7509640 1 --2- 172.21.15.49:0/1965867379 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fefd014c630 0x7fefd015c450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fefc8034040 tx=0x7fefc8004a40 comp rx=0 tx=0).stop 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.696+0000 7fefd5aa6640 1 -- 172.21.15.49:0/1965867379 reap_dead start 2024-05-09T08:58:41.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.697+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 shutdown_connections 2024-05-09T08:58:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.697+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 >> 172.21.15.49:0/1965867379 conn(0x7fefd00c8e00 msgr2=0x7fefd01513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.697+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 shutdown_connections 2024-05-09T08:58:41.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.697+0000 7fefd7509640 1 -- 172.21.15.49:0/1965867379 wait complete. 2024-05-09T08:58:41.707 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T08:58:41.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.814+0000 7fbea8b1b640 1 Processor -- start 2024-05-09T08:58:41.813 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.814+0000 7fbea8b1b640 1 -- start start 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea8b1b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbea4155080 0x7fbea4070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea8b1b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbea406b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea8b1b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea40c9fa0 con 0x7fbea406b740 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea8b1b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fbea40707f0 con 0x7fbea4155080 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea8b1b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbea40ca280 con 0x7fbea40c96f0 2024-05-09T08:58:41.816 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea2d9d640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fbea406b740 0x7fbea415e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:49794/0 (socket says 172.21.15.49:49794) 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea2d9d640 1 -- 172.21.15.49:0/2479750765 learned_addr learned my addr 172.21.15.49:0/2479750765 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea259c640 1 --2- 172.21.15.49:0/2479750765 >> v2:172.21.15.183:3300/0 conn(0x7fbea4155080 0x7fbea4070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbea406b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878189379 0 0) 0x7fbea40c9fa0 con 0x7fbea406b740 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.815+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe8c003520 con 0x7fbea406b740 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2362044825 0 0) 0x7fbe8c003520 con 0x7fbea406b740 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 msgr2=0x7fbea406b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbea406b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 >> v2:172.21.15.183:3300/0 conn(0x7fbea4155080 msgr2=0x7fbea4070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 --2- 172.21.15.49:0/2479750765 >> v2:172.21.15.183:3300/0 conn(0x7fbea4155080 0x7fbea4070140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea40ca560 con 0x7fbea406b740 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fbe9c0019f0 con 0x7fbea406b740 2024-05-09T08:58:41.817 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 >> v1:172.21.15.49:6789/0 conn(0x7fbea406b740 legacy=0x7fbea415e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbe8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbe8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe8c004d40 0x7fbe8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40707f0 con 0x7fbe8c0039b0 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40ca280 con 0x7fbe8c004d40 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c00b9c0 con 0x7fbea40c96f0 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbe8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbe8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea2d9d640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe8c004d40 0x7fbe8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 -- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 msgr2=0x7fbe8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea40c96f0 0x7fbe8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.818 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 -- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe8c004d40 msgr2=0x7fbe8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe8c004d40 0x7fbe8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.816+0000 7fbea259c640 1 -- 172.21.15.49:0/2479750765 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c003520 con 0x7fbe8c0039b0 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.817+0000 7fbea259c640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbe8c003dd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009ac0 tx=0x7fbe8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae64e084080f323b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.817+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94036070 con 0x7fbe8c0039b0 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.817+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe94031400 con 0x7fbe8c0039b0 2024-05-09T08:58:41.819 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.817+0000 7fbea159a640 1 -- 172.21.15.49:0/2479750765 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94009690 con 0x7fbe8c0039b0 2024-05-09T08:58:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 -- 172.21.15.49:0/2479750765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 msgr2=0x7fbe8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.822 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/2479750765 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbe8c003dd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009ac0 tx=0x7fbe8c005830 comp rx=0 tx=0).stop 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea2d9d640 1 -- 172.21.15.49:0/2479750765 reap_dead start 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 -- 172.21.15.49:0/2479750765 shutdown_connections 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 -- 172.21.15.49:0/2479750765 >> 172.21.15.49:0/2479750765 conn(0x7fbea4072230 msgr2=0x7fbea4072610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 -- 172.21.15.49:0/2479750765 shutdown_connections 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.818+0000 7fbea8b1b640 1 -- 172.21.15.49:0/2479750765 wait complete. 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.819+0000 7fbea8b1b640 1 Processor -- start 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.819+0000 7fbea8b1b640 1 -- start start 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 0x7fbea41f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 0x7fbea4150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4155460 con 0x7fbe8c0039b0 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbea40cab20 con 0x7fbea415dbe0 2024-05-09T08:58:41.823 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea8b1b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fbea406d130 con 0x7fbea4148300 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53124/0 (socket says 172.21.15.49:53124) 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 -- 172.21.15.49:0/1040217810 learned_addr learned my addr 172.21.15.49:0/1040217810 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 0x7fbea41f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea2d9d640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 0x7fbea4150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 -- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 msgr2=0x7fbea4150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 0x7fbea4150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 -- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 msgr2=0x7fbea41f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 0x7fbea41f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea259c640 1 -- 172.21.15.49:0/1040217810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea4153440 con 0x7fbe8c0039b0 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 0x7fbea41f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:41.824 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.820+0000 7fbea2d9d640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 0x7fbea4150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbea259c640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009be0 tx=0x7fbe94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94040020 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe94003950 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94036040 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea414eec0 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.821+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea414f480 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.822+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea406f1b0 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.822+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe94043020 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.823+0000 7fbe92ffd640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 0x7fbe8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.823+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fbe940ab0a0 con 0x7fbe8c0039b0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.823+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 0x7fbe8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:41.825 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.823+0000 7fbea1d9b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 0x7fbe8406e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbea414ed40 tx=0x7fbe98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:41.826 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:41.827+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbea406f1b0 con 0x7fbe8c0039b0 2024-05-09T08:58:42.010 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.011+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fbea404f660 con 0x7fbe8406be30 2024-05-09T08:58:42.011 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.012+0000 7fbe92ffd640 1 -- 172.21.15.49:0/1040217810 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbea404f660 con 0x7fbe8406be30 2024-05-09T08:58:42.011 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T08:58:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 msgr2=0x7fbe8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 0x7fbe8406e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbea414ed40 tx=0x7fbe98008040 comp rx=0 tx=0).stop 2024-05-09T08:58:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 msgr2=0x7fbea4150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.013 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009be0 tx=0x7fbe94004730 comp rx=0 tx=0).stop 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 shutdown_connections 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fbe8406be30 0x7fbe8406e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbea4148300 0x7fbea4150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbea415dbe0 0x7fbea41f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 --2- 172.21.15.49:0/1040217810 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fbe8c0039b0 0x7fbea4150260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 >> 172.21.15.49:0/1040217810 conn(0x7fbea4072230 msgr2=0x7fbea4152660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 shutdown_connections 2024-05-09T08:58:42.014 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.014+0000 7fbea8b1b640 1 -- 172.21.15.49:0/1040217810 wait complete. 2024-05-09T08:58:42.024 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:58:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.132+0000 7f3fe08f5640 1 Processor -- start 2024-05-09T08:58:42.131 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.132+0000 7f3fe08f5640 1 -- start start 2024-05-09T08:58:42.133 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3fdc1517f0 0x7f3fdc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3fdc0cd520 0x7f3fdc0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f3fdc151bd0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f3fdc15e260 con 0x7f3fdc1517f0 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fe08f5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc15e520 con 0x7f3fdc075200 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3fdc0cd520 0x7f3fdc0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f3fdc0cd520 0x7f3fdc0cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53126/0 (socket says 172.21.15.49:53126) 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 -- 172.21.15.49:0/1191161308 learned_addr learned my addr 172.21.15.49:0/1191161308 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fd9d9b640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fda59c640 1 --2- 172.21.15.49:0/1191161308 >> v2:172.21.15.183:3300/0 conn(0x7f3fdc1517f0 0x7f3fdc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 -- 172.21.15.49:0/1191161308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 msgr2=0x7f3fdc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:42.134 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 -- 172.21.15.49:0/1191161308 >> v2:172.21.15.183:3300/0 conn(0x7f3fdc1517f0 msgr2=0x7f3fdc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 --2- 172.21.15.49:0/1191161308 >> v2:172.21.15.183:3300/0 conn(0x7f3fdc1517f0 0x7f3fdc15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.133+0000 7f3fdad9d640 1 -- 172.21.15.49:0/1191161308 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc15e800 con 0x7f3fdc0cd520 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.134+0000 7f3fdad9d640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc0cc190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009e50 tx=0x7f3fd402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233cdd5c54deb809 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.134+0000 7f3fd959a640 1 -- 172.21.15.49:0/1191161308 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4036070 con 0x7f3fdc0cd520 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.134+0000 7f3fd959a640 1 -- 172.21.15.49:0/1191161308 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd4031600 con 0x7f3fdc0cd520 2024-05-09T08:58:42.135 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.134+0000 7f3fd959a640 1 -- 172.21.15.49:0/1191161308 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40319d0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 -- 172.21.15.49:0/1191161308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 msgr2=0x7f3fdc0cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc0cc190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009e50 tx=0x7f3fd402a750 comp rx=0 tx=0).stop 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 -- 172.21.15.49:0/1191161308 shutdown_connections 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc0cc190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/1191161308 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.138 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/1191161308 >> v2:172.21.15.183:3300/0 conn(0x7f3fdc1517f0 0x7f3fdc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 -- 172.21.15.49:0/1191161308 >> 172.21.15.49:0/1191161308 conn(0x7f3fdc06bd00 msgr2=0x7f3fdc074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 -- 172.21.15.49:0/1191161308 shutdown_connections 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.135+0000 7f3fe08f5640 1 -- 172.21.15.49:0/1191161308 wait complete. 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.136+0000 7f3fe08f5640 1 Processor -- start 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.136+0000 7f3fe08f5640 1 -- start start 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc1517f0 0x7f3fdc1e5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc15f0a0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc15edc0 con 0x7f3fdc1517f0 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fe08f5640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc0cbc70 con 0x7f3fdc075200 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.139 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:53140/0 (socket says 172.21.15.49:53140) 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 -- 172.21.15.49:0/728260461 learned_addr learned my addr 172.21.15.49:0/728260461 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fda59c640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fdad9d640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc1517f0 0x7f3fdc1e5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 -- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 msgr2=0x7f3fdc0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 -- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc1517f0 msgr2=0x7f3fdc1e5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc1517f0 0x7f3fdc1e5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.137+0000 7f3fd9d9b640 1 -- 172.21.15.49:0/728260461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc1f4ab0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fd9d9b640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000e890 tx=0x7f3fd000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0017070 con 0x7f3fdc0cd520 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd000c940 con 0x7f3fdc0cd520 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc0ca0d0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.140 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000cd10 con 0x7f3fdc0cd520 2024-05-09T08:58:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.138+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc0ca690 con 0x7f3fdc0cd520 2024-05-09T08:58:42.141 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.139+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fdc068560 con 0x7f3fdc0cd520 2024-05-09T08:58:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.139+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fd001a070 con 0x7f3fdc0cd520 2024-05-09T08:58:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.140+0000 7f3fcaffd640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 0x7f3fbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:58:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.140+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f3fd008b1d0 con 0x7f3fdc0cd520 2024-05-09T08:58:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.140+0000 7f3fda59c640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 0x7f3fbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:58:42.142 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.141+0000 7f3fda59c640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 0x7f3fbc06e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009a30 tx=0x7f3fcc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:58:42.143 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.144+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3fd0059b60 con 0x7f3fdc0cd520 2024-05-09T08:58:42.323 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.324+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3fdc1f4d90 con 0x7f3fbc06be30 2024-05-09T08:58:42.330 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.331+0000 7f3fcaffd640 1 -- 172.21.15.49:0/728260461 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252462 (secure 0 0 0) 0x7f3fdc1f4d90 con 0x7f3fbc06be30 2024-05-09T08:58:42.335 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:58:42.337 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:58:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 msgr2=0x7f3fbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.340 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 0x7f3fbc06e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009a30 tx=0x7f3fcc008040 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 msgr2=0x7f3fdc1f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000e890 tx=0x7f3fd000ed50 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 shutdown_connections 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f3fbc06be30 0x7f3fbc06e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fdc1517f0 0x7f3fdc1e5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f3fdc0cd520 0x7f3fdc1f4110 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 --2- 172.21.15.49:0/728260461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3fdc075200 0x7f3fdc0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.333+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 >> 172.21.15.49:0/728260461 conn(0x7f3fdc06bd00 msgr2=0x7f3fdc0b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:58:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.334+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 shutdown_connections 2024-05-09T08:58:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:58:42.334+0000 7f3fe08f5640 1 -- 172.21.15.49:0/728260461 wait complete. 2024-05-09T08:58:42.344 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-05-09T08:58:40.605083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634963,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":410,"num_write_kb":2540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":498,"ondisk_log_size":498,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239564,"kb_used_data":9136,"kb_used_omap":24,"kb_used_meta":230311,"kb_avail":754735156,"statfs":{"total":773094113280,"available":772848799744,"internally_reserved":0,"allocated":9355264,"data_stored":4932258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":235838536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":314,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.212579"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395148+0000","last_change":"2024-05-09T08:56:01.699813+0000","last_active":"2024-05-09T08:58:30.395148+0000","last_peered":"2024-05-09T08:58:30.395148+0000","last_clean":"2024-05-09T08:58:30.395148+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:30.395148+0000","last_undegraded":"2024-05-09T08:58:30.395148+0000","last_fullsized":"2024-05-09T08:58:30.395148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:44.803944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501060+0000","last_change":"2024-05-09T08:58:14.269933+0000","last_active":"2024-05-09T08:58:32.501060+0000","last_peered":"2024-05-09T08:58:32.501060+0000","last_clean":"2024-05-09T08:58:32.501060+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:32.501060+0000","last_undegraded":"2024-05-09T08:58:32.501060+0000","last_fullsized":"2024-05-09T08:58:32.501060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:04.959503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"83'3","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606913+0000","last_change":"2024-05-09T08:58:14.263652+0000","last_active":"2024-05-09T08:58:35.606913+0000","last_peered":"2024-05-09T08:58:35.606913+0000","last_clean":"2024-05-09T08:58:35.606913+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:35.606913+0000","last_undegraded":"2024-05-09T08:58:35.606913+0000","last_fullsized":"2024-05-09T08:58:35.606913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:26:38.362008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.629232+0000","last_change":"2024-05-09T08:58:13.288413+0000","last_active":"2024-05-09T08:58:35.629232+0000","last_peered":"2024-05-09T08:58:35.629232+0000","last_clean":"2024-05-09T08:58:35.629232+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:35.629232+0000","last_undegraded":"2024-05-09T08:58:35.629232+0000","last_fullsized":"2024-05-09T08:58:35.629232+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:35:09.200128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"83'1","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.601095+0000","last_change":"2024-05-09T08:58:13.296978+0000","last_active":"2024-05-09T08:58:35.601095+0000","last_peered":"2024-05-09T08:58:35.601095+0000","last_clean":"2024-05-09T08:58:35.601095+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:35.601095+0000","last_undegraded":"2024-05-09T08:58:35.601095+0000","last_fullsized":"2024-05-09T08:58:35.601095+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:32:37.092057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'1","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395562+0000","last_change":"2024-05-09T08:58:24.726155+0000","last_active":"2024-05-09T08:58:30.395562+0000","last_peered":"2024-05-09T08:58:30.395562+0000","last_clean":"2024-05-09T08:58:30.395562+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:30.395562+0000","last_undegraded":"2024-05-09T08:58:30.395562+0000","last_fullsized":"2024-05-09T08:58:30.395562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.468307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":57,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.621069+0000","last_change":"2024-05-09T08:58:14.264161+0000","last_active":"2024-05-09T08:58:35.621069+0000","last_peered":"2024-05-09T08:58:35.621069+0000","last_clean":"2024-05-09T08:58:35.621069+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:35.621069+0000","last_undegraded":"2024-05-09T08:58:35.621069+0000","last_fullsized":"2024-05-09T08:58:35.621069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:29:36.088586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"83'5","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396779+0000","last_change":"2024-05-09T08:56:01.700021+0000","last_active":"2024-05-09T08:58:30.396779+0000","last_peered":"2024-05-09T08:58:30.396779+0000","last_clean":"2024-05-09T08:58:30.396779+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:30.396779+0000","last_undegraded":"2024-05-09T08:58:30.396779+0000","last_fullsized":"2024-05-09T08:58:30.396779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:18.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.606929+0000","last_change":"2024-05-09T08:56:01.700598+0000","last_active":"2024-05-09T08:58:30.606929+0000","last_peered":"2024-05-09T08:58:30.606929+0000","last_clean":"2024-05-09T08:58:30.606929+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:30.606929+0000","last_undegraded":"2024-05-09T08:58:30.606929+0000","last_fullsized":"2024-05-09T08:58:30.606929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:57:29.474863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217571+0000","last_change":"2024-05-09T08:56:01.694383+0000","last_active":"2024-05-09T08:58:33.217571+0000","last_peered":"2024-05-09T08:58:33.217571+0000","last_clean":"2024-05-09T08:58:33.217571+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:33.217571+0000","last_undegraded":"2024-05-09T08:58:33.217571+0000","last_fullsized":"2024-05-09T08:58:33.217571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:49:21.167259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217478+0000","last_change":"2024-05-09T08:56:01.694079+0000","last_active":"2024-05-09T08:58:33.217478+0000","last_peered":"2024-05-09T08:58:33.217478+0000","last_clean":"2024-05-09T08:58:33.217478+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:33.217478+0000","last_undegraded":"2024-05-09T08:58:33.217478+0000","last_fullsized":"2024-05-09T08:58:33.217478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:01:29.834947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'4","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395143+0000","last_change":"2024-05-09T08:56:01.698258+0000","last_active":"2024-05-09T08:58:30.395143+0000","last_peered":"2024-05-09T08:58:30.395143+0000","last_clean":"2024-05-09T08:58:30.395143+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:30.395143+0000","last_undegraded":"2024-05-09T08:58:30.395143+0000","last_fullsized":"2024-05-09T08:58:30.395143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:03.325644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397201+0000","last_change":"2024-05-09T08:56:01.700824+0000","last_active":"2024-05-09T08:58:30.397201+0000","last_peered":"2024-05-09T08:58:30.397201+0000","last_clean":"2024-05-09T08:58:30.397201+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:30.397201+0000","last_undegraded":"2024-05-09T08:58:30.397201+0000","last_fullsized":"2024-05-09T08:58:30.397201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:05:14.675269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393293+0000","last_change":"2024-05-09T08:58:14.292202+0000","last_active":"2024-05-09T08:58:30.393293+0000","last_peered":"2024-05-09T08:58:30.393293+0000","last_clean":"2024-05-09T08:58:30.393293+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:30.393293+0000","last_undegraded":"2024-05-09T08:58:30.393293+0000","last_fullsized":"2024-05-09T08:58:30.393293+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:36.668146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'3","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500833+0000","last_change":"2024-05-09T08:56:01.687580+0000","last_active":"2024-05-09T08:58:32.500833+0000","last_peered":"2024-05-09T08:58:32.500833+0000","last_clean":"2024-05-09T08:58:32.500833+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:32.500833+0000","last_undegraded":"2024-05-09T08:58:32.500833+0000","last_fullsized":"2024-05-09T08:58:32.500833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:48.498156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395984+0000","last_change":"2024-05-09T08:58:24.725815+0000","last_active":"2024-05-09T08:58:30.395984+0000","last_peered":"2024-05-09T08:58:30.395984+0000","last_clean":"2024-05-09T08:58:30.395984+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:30.395984+0000","last_undegraded":"2024-05-09T08:58:30.395984+0000","last_fullsized":"2024-05-09T08:58:30.395984+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:57.983863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396315+0000","last_change":"2024-05-09T08:56:01.699404+0000","last_active":"2024-05-09T08:58:30.396315+0000","last_peered":"2024-05-09T08:58:30.396315+0000","last_clean":"2024-05-09T08:58:30.396315+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:30.396315+0000","last_undegraded":"2024-05-09T08:58:30.396315+0000","last_fullsized":"2024-05-09T08:58:30.396315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:48:49.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":63,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.623621+0000","last_change":"2024-05-09T08:58:13.290857+0000","last_active":"2024-05-09T08:58:35.623621+0000","last_peered":"2024-05-09T08:58:35.623621+0000","last_clean":"2024-05-09T08:58:35.623621+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:35.623621+0000","last_undegraded":"2024-05-09T08:58:35.623621+0000","last_fullsized":"2024-05-09T08:58:35.623621+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:02:13.212701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395975+0000","last_change":"2024-05-09T08:58:13.287132+0000","last_active":"2024-05-09T08:58:30.395975+0000","last_peered":"2024-05-09T08:58:30.395975+0000","last_clean":"2024-05-09T08:58:30.395975+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:30.395975+0000","last_undegraded":"2024-05-09T08:58:30.395975+0000","last_fullsized":"2024-05-09T08:58:30.395975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:56:04.524395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"83'7","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.603643+0000","last_change":"2024-05-09T08:56:01.688937+0000","last_active":"2024-05-09T08:58:35.603643+0000","last_peered":"2024-05-09T08:58:35.603643+0000","last_clean":"2024-05-09T08:58:35.603643+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:35.603643+0000","last_undegraded":"2024-05-09T08:58:35.603643+0000","last_fullsized":"2024-05-09T08:58:35.603643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:36.603904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393563+0000","last_change":"2024-05-09T08:58:14.267612+0000","last_active":"2024-05-09T08:58:30.393563+0000","last_peered":"2024-05-09T08:58:30.393563+0000","last_clean":"2024-05-09T08:58:30.393563+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:30.393563+0000","last_undegraded":"2024-05-09T08:58:30.393563+0000","last_fullsized":"2024-05-09T08:58:30.393563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:37:58.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395620+0000","last_change":"2024-05-09T08:56:01.699521+0000","last_active":"2024-05-09T08:58:30.395620+0000","last_peered":"2024-05-09T08:58:30.395620+0000","last_clean":"2024-05-09T08:58:30.395620+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:30.395620+0000","last_undegraded":"2024-05-09T08:58:30.395620+0000","last_fullsized":"2024-05-09T08:58:30.395620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:33:33.999801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396903+0000","last_change":"2024-05-09T08:56:01.694637+0000","last_active":"2024-05-09T08:58:30.396903+0000","last_peered":"2024-05-09T08:58:30.396903+0000","last_clean":"2024-05-09T08:58:30.396903+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:30.396903+0000","last_undegraded":"2024-05-09T08:58:30.396903+0000","last_fullsized":"2024-05-09T08:58:30.396903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:45.061316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396093+0000","last_change":"2024-05-09T08:56:01.700667+0000","last_active":"2024-05-09T08:58:30.396093+0000","last_peered":"2024-05-09T08:58:30.396093+0000","last_clean":"2024-05-09T08:58:30.396093+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:30.396093+0000","last_undegraded":"2024-05-09T08:58:30.396093+0000","last_fullsized":"2024-05-09T08:58:30.396093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:57:28.917778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397209+0000","last_change":"2024-05-09T08:56:01.694446+0000","last_active":"2024-05-09T08:58:30.397209+0000","last_peered":"2024-05-09T08:58:30.397209+0000","last_clean":"2024-05-09T08:58:30.397209+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:30.397209+0000","last_undegraded":"2024-05-09T08:58:30.397209+0000","last_fullsized":"2024-05-09T08:58:30.397209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:09.472409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217614+0000","last_change":"2024-05-09T08:56:01.697464+0000","last_active":"2024-05-09T08:58:33.217614+0000","last_peered":"2024-05-09T08:58:33.217614+0000","last_clean":"2024-05-09T08:58:33.217614+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:33.217614+0000","last_undegraded":"2024-05-09T08:58:33.217614+0000","last_fullsized":"2024-05-09T08:58:33.217614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:32.179135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'3","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605574+0000","last_change":"2024-05-09T08:56:01.694284+0000","last_active":"2024-05-09T08:58:35.605574+0000","last_peered":"2024-05-09T08:58:35.605574+0000","last_clean":"2024-05-09T08:58:35.605574+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:35.605574+0000","last_undegraded":"2024-05-09T08:58:35.605574+0000","last_fullsized":"2024-05-09T08:58:35.605574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:05:41.094533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501562+0000","last_change":"2024-05-09T08:56:50.303600+0000","last_active":"2024-05-09T08:58:32.501562+0000","last_peered":"2024-05-09T08:58:32.501562+0000","last_clean":"2024-05-09T08:58:32.501562+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:32.501562+0000","last_undegraded":"2024-05-09T08:58:32.501562+0000","last_fullsized":"2024-05-09T08:58:32.501562+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:01.056599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"83'5","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.604133+0000","last_change":"2024-05-09T08:58:14.265053+0000","last_active":"2024-05-09T08:58:35.604133+0000","last_peered":"2024-05-09T08:58:35.604133+0000","last_clean":"2024-05-09T08:58:35.604133+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:35.604133+0000","last_undegraded":"2024-05-09T08:58:35.604133+0000","last_fullsized":"2024-05-09T08:58:35.604133+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:16:26.385704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'4","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.603005+0000","last_change":"2024-05-09T08:58:24.616702+0000","last_active":"2024-05-09T08:58:35.603005+0000","last_peered":"2024-05-09T08:58:35.603005+0000","last_clean":"2024-05-09T08:58:35.603005+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:35.603005+0000","last_undegraded":"2024-05-09T08:58:35.603005+0000","last_fullsized":"2024-05-09T08:58:35.603005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:24:31.696751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":65,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.633084+0000","last_change":"2024-05-09T08:58:13.289032+0000","last_active":"2024-05-09T08:58:35.633084+0000","last_peered":"2024-05-09T08:58:35.633084+0000","last_clean":"2024-05-09T08:58:35.633084+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:35.633084+0000","last_undegraded":"2024-05-09T08:58:35.633084+0000","last_fullsized":"2024-05-09T08:58:35.633084+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:08:41.346657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606282+0000","last_change":"2024-05-09T08:58:14.271872+0000","last_active":"2024-05-09T08:58:35.606282+0000","last_peered":"2024-05-09T08:58:35.606282+0000","last_clean":"2024-05-09T08:58:35.606282+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:35.606282+0000","last_undegraded":"2024-05-09T08:58:35.606282+0000","last_fullsized":"2024-05-09T08:58:35.606282+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:46.256437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606072+0000","last_change":"2024-05-09T08:56:01.689102+0000","last_active":"2024-05-09T08:58:35.606072+0000","last_peered":"2024-05-09T08:58:35.606072+0000","last_clean":"2024-05-09T08:58:35.606072+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:35.606072+0000","last_undegraded":"2024-05-09T08:58:35.606072+0000","last_fullsized":"2024-05-09T08:58:35.606072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:42.826347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397438+0000","last_change":"2024-05-09T08:56:00.694108+0000","last_active":"2024-05-09T08:58:30.397438+0000","last_peered":"2024-05-09T08:58:30.397438+0000","last_clean":"2024-05-09T08:58:30.397438+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:30.397438+0000","last_undegraded":"2024-05-09T08:58:30.397438+0000","last_fullsized":"2024-05-09T08:58:30.397438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:12.087640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"83'3","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393513+0000","last_change":"2024-05-09T08:58:14.261772+0000","last_active":"2024-05-09T08:58:30.393513+0000","last_peered":"2024-05-09T08:58:30.393513+0000","last_clean":"2024-05-09T08:58:30.393513+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:30.393513+0000","last_undegraded":"2024-05-09T08:58:30.393513+0000","last_fullsized":"2024-05-09T08:58:30.393513+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:14:37.978589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'7","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216857+0000","last_change":"2024-05-09T08:56:00.688045+0000","last_active":"2024-05-09T08:58:33.216857+0000","last_peered":"2024-05-09T08:58:33.216857+0000","last_clean":"2024-05-09T08:58:33.216857+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:33.216857+0000","last_undegraded":"2024-05-09T08:58:33.216857+0000","last_fullsized":"2024-05-09T08:58:33.216857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:35.805073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396930+0000","last_change":"2024-05-09T08:56:00.692099+0000","last_active":"2024-05-09T08:58:30.396930+0000","last_peered":"2024-05-09T08:58:30.396930+0000","last_clean":"2024-05-09T08:58:30.396930+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:30.396930+0000","last_undegraded":"2024-05-09T08:58:30.396930+0000","last_fullsized":"2024-05-09T08:58:30.396930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:43.850485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"83'4","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605756+0000","last_change":"2024-05-09T08:58:14.271818+0000","last_active":"2024-05-09T08:58:35.605756+0000","last_peered":"2024-05-09T08:58:35.605756+0000","last_clean":"2024-05-09T08:58:35.605756+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:35.605756+0000","last_undegraded":"2024-05-09T08:58:35.605756+0000","last_fullsized":"2024-05-09T08:58:35.605756+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:09:55.613511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"20'3","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396200+0000","last_change":"2024-05-09T08:56:00.682908+0000","last_active":"2024-05-09T08:58:30.396200+0000","last_peered":"2024-05-09T08:58:30.396200+0000","last_clean":"2024-05-09T08:58:30.396200+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:30.396200+0000","last_undegraded":"2024-05-09T08:58:30.396200+0000","last_fullsized":"2024-05-09T08:58:30.396200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:11:03.540100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500695+0000","last_change":"2024-05-09T08:56:01.696742+0000","last_active":"2024-05-09T08:58:32.500695+0000","last_peered":"2024-05-09T08:58:32.500695+0000","last_clean":"2024-05-09T08:58:32.500695+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:32.500695+0000","last_undegraded":"2024-05-09T08:58:32.500695+0000","last_fullsized":"2024-05-09T08:58:32.500695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:19.938580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'7","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217039+0000","last_change":"2024-05-09T08:56:00.689757+0000","last_active":"2024-05-09T08:58:33.217039+0000","last_peered":"2024-05-09T08:58:33.217039+0000","last_clean":"2024-05-09T08:58:33.217039+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:33.217039+0000","last_undegraded":"2024-05-09T08:58:33.217039+0000","last_fullsized":"2024-05-09T08:58:33.217039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:12:38.431738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395944+0000","last_change":"2024-05-09T08:56:01.699336+0000","last_active":"2024-05-09T08:58:30.395944+0000","last_peered":"2024-05-09T08:58:30.395944+0000","last_clean":"2024-05-09T08:58:30.395944+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:30.395944+0000","last_undegraded":"2024-05-09T08:58:30.395944+0000","last_fullsized":"2024-05-09T08:58:30.395944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:30:52.145283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395993+0000","last_change":"2024-05-09T08:56:00.684755+0000","last_active":"2024-05-09T08:58:30.395993+0000","last_peered":"2024-05-09T08:58:30.395993+0000","last_clean":"2024-05-09T08:58:30.395993+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:30.395993+0000","last_undegraded":"2024-05-09T08:58:30.395993+0000","last_fullsized":"2024-05-09T08:58:30.395993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:48:27.108957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393622+0000","last_change":"2024-05-09T08:58:14.267548+0000","last_active":"2024-05-09T08:58:30.393622+0000","last_peered":"2024-05-09T08:58:30.393622+0000","last_clean":"2024-05-09T08:58:30.393622+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:30.393622+0000","last_undegraded":"2024-05-09T08:58:30.393622+0000","last_fullsized":"2024-05-09T08:58:30.393622+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:34.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"63'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501356+0000","last_change":"2024-05-09T08:56:00.691233+0000","last_active":"2024-05-09T08:58:32.501356+0000","last_peered":"2024-05-09T08:58:32.501356+0000","last_clean":"2024-05-09T08:58:32.501356+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:32.501356+0000","last_undegraded":"2024-05-09T08:58:32.501356+0000","last_fullsized":"2024-05-09T08:58:32.501356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:53:06.989366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"83'8","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.607701+0000","last_change":"2024-05-09T08:56:01.695701+0000","last_active":"2024-05-09T08:58:35.607701+0000","last_peered":"2024-05-09T08:58:35.607701+0000","last_clean":"2024-05-09T08:58:35.607701+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:35.607701+0000","last_undegraded":"2024-05-09T08:58:35.607701+0000","last_fullsized":"2024-05-09T08:58:35.607701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:44:17.348542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":165,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.589905+0000","last_change":"2024-05-09T08:56:00.696993+0000","last_active":"2024-05-09T08:58:35.589905+0000","last_peered":"2024-05-09T08:58:35.589905+0000","last_clean":"2024-05-09T08:58:35.589905+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:35.589905+0000","last_undegraded":"2024-05-09T08:58:35.589905+0000","last_fullsized":"2024-05-09T08:58:35.589905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:10:16.612335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37473,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395893+0000","last_change":"2024-05-09T08:56:01.697565+0000","last_active":"2024-05-09T08:58:30.395893+0000","last_peered":"2024-05-09T08:58:30.395893+0000","last_clean":"2024-05-09T08:58:30.395893+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:30.395893+0000","last_undegraded":"2024-05-09T08:58:30.395893+0000","last_fullsized":"2024-05-09T08:58:30.395893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:34:09.210770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'9","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393457+0000","last_change":"2024-05-09T08:58:14.282091+0000","last_active":"2024-05-09T08:58:30.393457+0000","last_peered":"2024-05-09T08:58:30.393457+0000","last_clean":"2024-05-09T08:58:30.393457+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:30.393457+0000","last_undegraded":"2024-05-09T08:58:30.393457+0000","last_fullsized":"2024-05-09T08:58:30.393457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:22.711934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.265567+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.863184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"63'6","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395757+0000","last_change":"2024-05-09T08:56:00.691990+0000","last_active":"2024-05-09T08:58:30.395757+0000","last_peered":"2024-05-09T08:58:30.395757+0000","last_clean":"2024-05-09T08:58:30.395757+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:30.395757+0000","last_undegraded":"2024-05-09T08:58:30.395757+0000","last_fullsized":"2024-05-09T08:58:30.395757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:27:23.829562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395739+0000","last_change":"2024-05-09T08:56:01.699653+0000","last_active":"2024-05-09T08:58:30.395739+0000","last_peered":"2024-05-09T08:58:30.395739+0000","last_clean":"2024-05-09T08:58:30.395739+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:30.395739+0000","last_undegraded":"2024-05-09T08:58:30.395739+0000","last_fullsized":"2024-05-09T08:58:30.395739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:38.593237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"20'5","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216858+0000","last_change":"2024-05-09T08:56:00.688176+0000","last_active":"2024-05-09T08:58:33.216858+0000","last_peered":"2024-05-09T08:58:33.216858+0000","last_clean":"2024-05-09T08:58:33.216858+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:33.216858+0000","last_undegraded":"2024-05-09T08:58:33.216858+0000","last_fullsized":"2024-05-09T08:58:33.216858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:52.913509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606462+0000","last_change":"2024-05-09T08:58:14.272036+0000","last_active":"2024-05-09T08:58:35.606462+0000","last_peered":"2024-05-09T08:58:35.606462+0000","last_clean":"2024-05-09T08:58:35.606462+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:35.606462+0000","last_undegraded":"2024-05-09T08:58:35.606462+0000","last_fullsized":"2024-05-09T08:58:35.606462+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:53:02.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'17","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395099+0000","last_change":"2024-05-09T08:58:13.285688+0000","last_active":"2024-05-09T08:58:30.395099+0000","last_peered":"2024-05-09T08:58:30.395099+0000","last_clean":"2024-05-09T08:58:30.395099+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:30.395099+0000","last_undegraded":"2024-05-09T08:58:30.395099+0000","last_fullsized":"2024-05-09T08:58:30.395099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:47:08.347374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394658+0000","last_change":"2024-05-09T08:56:01.695060+0000","last_active":"2024-05-09T08:58:30.394658+0000","last_peered":"2024-05-09T08:58:30.394658+0000","last_clean":"2024-05-09T08:58:30.394658+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:30.394658+0000","last_undegraded":"2024-05-09T08:58:30.394658+0000","last_fullsized":"2024-05-09T08:58:30.394658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.524405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'2","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394837+0000","last_change":"2024-05-09T08:58:14.279415+0000","last_active":"2024-05-09T08:58:30.394837+0000","last_peered":"2024-05-09T08:58:30.394837+0000","last_clean":"2024-05-09T08:58:30.394837+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:30.394837+0000","last_undegraded":"2024-05-09T08:58:30.394837+0000","last_fullsized":"2024-05-09T08:58:30.394837+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:04:14.922686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394991+0000","last_change":"2024-05-09T08:56:01.702967+0000","last_active":"2024-05-09T08:58:30.394991+0000","last_peered":"2024-05-09T08:58:30.394991+0000","last_clean":"2024-05-09T08:58:30.394991+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:30.394991+0000","last_undegraded":"2024-05-09T08:58:30.394991+0000","last_fullsized":"2024-05-09T08:58:30.394991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:27:47.149731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'5","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.398065+0000","last_change":"2024-05-09T08:58:30.398065+0000","last_active":"2024-05-09T08:58:30.398065+0000","last_peered":"2024-05-09T08:58:30.398065+0000","last_clean":"2024-05-09T08:58:30.398065+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:30.398065+0000","last_undegraded":"2024-05-09T08:58:30.398065+0000","last_fullsized":"2024-05-09T08:58:30.398065+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:53.076973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395324+0000","last_change":"2024-05-09T08:56:01.695685+0000","last_active":"2024-05-09T08:58:30.395324+0000","last_peered":"2024-05-09T08:58:30.395324+0000","last_clean":"2024-05-09T08:58:30.395324+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:30.395324+0000","last_undegraded":"2024-05-09T08:58:30.395324+0000","last_fullsized":"2024-05-09T08:58:30.395324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:40:09.799392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501455+0000","last_change":"2024-05-09T08:56:00.695635+0000","last_active":"2024-05-09T08:58:32.501455+0000","last_peered":"2024-05-09T08:58:32.501455+0000","last_clean":"2024-05-09T08:58:32.501455+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:32.501455+0000","last_undegraded":"2024-05-09T08:58:32.501455+0000","last_fullsized":"2024-05-09T08:58:32.501455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:01:46.503325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395654+0000","last_change":"2024-05-09T08:56:01.695721+0000","last_active":"2024-05-09T08:58:30.395654+0000","last_peered":"2024-05-09T08:58:30.395654+0000","last_clean":"2024-05-09T08:58:30.395654+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:30.395654+0000","last_undegraded":"2024-05-09T08:58:30.395654+0000","last_fullsized":"2024-05-09T08:58:30.395654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:19:22.926572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"20'5","reported_seq":64,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395180+0000","last_change":"2024-05-09T08:58:13.288568+0000","last_active":"2024-05-09T08:58:30.395180+0000","last_peered":"2024-05-09T08:58:30.395180+0000","last_clean":"2024-05-09T08:58:30.395180+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:30.395180+0000","last_undegraded":"2024-05-09T08:58:30.395180+0000","last_fullsized":"2024-05-09T08:58:30.395180+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:31:26.070716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395860+0000","last_change":"2024-05-09T08:56:01.700810+0000","last_active":"2024-05-09T08:58:30.395860+0000","last_peered":"2024-05-09T08:58:30.395860+0000","last_clean":"2024-05-09T08:58:30.395860+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:30.395860+0000","last_undegraded":"2024-05-09T08:58:30.395860+0000","last_fullsized":"2024-05-09T08:58:30.395860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:24.738490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395604+0000","last_change":"2024-05-09T08:58:17.324441+0000","last_active":"2024-05-09T08:58:30.395604+0000","last_peered":"2024-05-09T08:58:30.395604+0000","last_clean":"2024-05-09T08:58:30.395604+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:30.395604+0000","last_undegraded":"2024-05-09T08:58:30.395604+0000","last_fullsized":"2024-05-09T08:58:30.395604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:56.645068+0000","last_clean_scrub_stamp":"2024-05-09T08:55:56.645068+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:00:26.859077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396326+0000","last_change":"2024-05-09T08:58:13.297053+0000","last_active":"2024-05-09T08:58:30.396326+0000","last_peered":"2024-05-09T08:58:30.396326+0000","last_clean":"2024-05-09T08:58:30.396326+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:30.396326+0000","last_undegraded":"2024-05-09T08:58:30.396326+0000","last_fullsized":"2024-05-09T08:58:30.396326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:22.189205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397111+0000","last_change":"2024-05-09T08:56:01.700805+0000","last_active":"2024-05-09T08:58:30.397111+0000","last_peered":"2024-05-09T08:58:30.397111+0000","last_clean":"2024-05-09T08:58:30.397111+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:30.397111+0000","last_undegraded":"2024-05-09T08:58:30.397111+0000","last_fullsized":"2024-05-09T08:58:30.397111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:02.083719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395565+0000","last_change":"2024-05-09T08:56:00.697542+0000","last_active":"2024-05-09T08:58:30.395565+0000","last_peered":"2024-05-09T08:58:30.395565+0000","last_clean":"2024-05-09T08:58:30.395565+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:30.395565+0000","last_undegraded":"2024-05-09T08:58:30.395565+0000","last_fullsized":"2024-05-09T08:58:30.395565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:30.560527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393134+0000","last_change":"2024-05-09T08:58:14.260638+0000","last_active":"2024-05-09T08:58:30.393134+0000","last_peered":"2024-05-09T08:58:30.393134+0000","last_clean":"2024-05-09T08:58:30.393134+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:30.393134+0000","last_undegraded":"2024-05-09T08:58:30.393134+0000","last_fullsized":"2024-05-09T08:58:30.393134+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:13:50.474389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'7","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395116+0000","last_change":"2024-05-09T08:58:13.299307+0000","last_active":"2024-05-09T08:58:30.395116+0000","last_peered":"2024-05-09T08:58:30.395116+0000","last_clean":"2024-05-09T08:58:30.395116+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:30.395116+0000","last_undegraded":"2024-05-09T08:58:30.395116+0000","last_fullsized":"2024-05-09T08:58:30.395116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:10:01.213138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"83'3","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.502038+0000","last_change":"2024-05-09T08:58:13.287477+0000","last_active":"2024-05-09T08:58:32.502038+0000","last_peered":"2024-05-09T08:58:32.502038+0000","last_clean":"2024-05-09T08:58:32.502038+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:32.502038+0000","last_undegraded":"2024-05-09T08:58:32.502038+0000","last_fullsized":"2024-05-09T08:58:32.502038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:01:47.092823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"20'5","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395819+0000","last_change":"2024-05-09T08:58:24.363355+0000","last_active":"2024-05-09T08:58:30.395819+0000","last_peered":"2024-05-09T08:58:30.395819+0000","last_clean":"2024-05-09T08:58:30.395819+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:30.395819+0000","last_undegraded":"2024-05-09T08:58:30.395819+0000","last_fullsized":"2024-05-09T08:58:30.395819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:08:28.876851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.605857+0000","last_change":"2024-05-09T08:58:14.267021+0000","last_active":"2024-05-09T08:58:35.605857+0000","last_peered":"2024-05-09T08:58:35.605857+0000","last_clean":"2024-05-09T08:58:35.605857+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:35.605857+0000","last_undegraded":"2024-05-09T08:58:35.605857+0000","last_fullsized":"2024-05-09T08:58:35.605857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:15.842389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501749+0000","last_change":"2024-05-09T08:58:14.270964+0000","last_active":"2024-05-09T08:58:32.501749+0000","last_peered":"2024-05-09T08:58:32.501749+0000","last_clean":"2024-05-09T08:58:32.501749+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:32.501749+0000","last_undegraded":"2024-05-09T08:58:32.501749+0000","last_fullsized":"2024-05-09T08:58:32.501749+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:52:33.749350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396574+0000","last_change":"2024-05-09T08:56:01.694329+0000","last_active":"2024-05-09T08:58:30.396574+0000","last_peered":"2024-05-09T08:58:30.396574+0000","last_clean":"2024-05-09T08:58:30.396574+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:30.396574+0000","last_undegraded":"2024-05-09T08:58:30.396574+0000","last_fullsized":"2024-05-09T08:58:30.396574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:57.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"20'4","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396068+0000","last_change":"2024-05-09T08:58:14.276305+0000","last_active":"2024-05-09T08:58:30.396068+0000","last_peered":"2024-05-09T08:58:30.396068+0000","last_clean":"2024-05-09T08:58:30.396068+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:30.396068+0000","last_undegraded":"2024-05-09T08:58:30.396068+0000","last_fullsized":"2024-05-09T08:58:30.396068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:47:02.067941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501652+0000","last_change":"2024-05-09T08:58:13.286175+0000","last_active":"2024-05-09T08:58:32.501652+0000","last_peered":"2024-05-09T08:58:32.501652+0000","last_clean":"2024-05-09T08:58:32.501652+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:32.501652+0000","last_undegraded":"2024-05-09T08:58:32.501652+0000","last_fullsized":"2024-05-09T08:58:32.501652+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:54:03.525310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"64'16","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.501154+0000","last_change":"2024-05-09T08:56:00.690776+0000","last_active":"2024-05-09T08:58:32.501154+0000","last_peered":"2024-05-09T08:58:32.501154+0000","last_clean":"2024-05-09T08:58:32.501154+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:32.501154+0000","last_undegraded":"2024-05-09T08:58:32.501154+0000","last_fullsized":"2024-05-09T08:58:32.501154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:15:48.513164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"83'1","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397253+0000","last_change":"2024-05-09T08:56:01.697747+0000","last_active":"2024-05-09T08:58:30.397253+0000","last_peered":"2024-05-09T08:58:30.397253+0000","last_clean":"2024-05-09T08:58:30.397253+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:30.397253+0000","last_undegraded":"2024-05-09T08:58:30.397253+0000","last_fullsized":"2024-05-09T08:58:30.397253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:17.983516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"63'8","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.216985+0000","last_change":"2024-05-09T08:56:00.693900+0000","last_active":"2024-05-09T08:58:33.216985+0000","last_peered":"2024-05-09T08:58:33.216985+0000","last_clean":"2024-05-09T08:58:33.216985+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:33.216985+0000","last_undegraded":"2024-05-09T08:58:33.216985+0000","last_fullsized":"2024-05-09T08:58:33.216985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:38:00.539764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393608+0000","last_change":"2024-05-09T08:58:14.266765+0000","last_active":"2024-05-09T08:58:30.393608+0000","last_peered":"2024-05-09T08:58:30.393608+0000","last_clean":"2024-05-09T08:58:30.393608+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:30.393608+0000","last_undegraded":"2024-05-09T08:58:30.393608+0000","last_fullsized":"2024-05-09T08:58:30.393608+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:42:33.135225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'2","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395494+0000","last_change":"2024-05-09T08:58:14.281333+0000","last_active":"2024-05-09T08:58:30.395494+0000","last_peered":"2024-05-09T08:58:30.395494+0000","last_clean":"2024-05-09T08:58:30.395494+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:30.395494+0000","last_undegraded":"2024-05-09T08:58:30.395494+0000","last_fullsized":"2024-05-09T08:58:30.395494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:19:46.445301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"83'1","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394689+0000","last_change":"2024-05-09T08:58:13.297000+0000","last_active":"2024-05-09T08:58:30.394689+0000","last_peered":"2024-05-09T08:58:30.394689+0000","last_clean":"2024-05-09T08:58:30.394689+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:30.394689+0000","last_undegraded":"2024-05-09T08:58:30.394689+0000","last_fullsized":"2024-05-09T08:58:30.394689+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:26.608081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"20'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500883+0000","last_change":"2024-05-09T08:56:00.693014+0000","last_active":"2024-05-09T08:58:32.500883+0000","last_peered":"2024-05-09T08:58:32.500883+0000","last_clean":"2024-05-09T08:58:32.500883+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:32.500883+0000","last_undegraded":"2024-05-09T08:58:32.500883+0000","last_fullsized":"2024-05-09T08:58:32.500883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:35:57.931712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"83'6","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.396942+0000","last_change":"2024-05-09T08:56:01.693114+0000","last_active":"2024-05-09T08:58:30.396942+0000","last_peered":"2024-05-09T08:58:30.396942+0000","last_clean":"2024-05-09T08:58:30.396942+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:30.396942+0000","last_undegraded":"2024-05-09T08:58:30.396942+0000","last_fullsized":"2024-05-09T08:58:30.396942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:10.777947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'4","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:32.500573+0000","last_change":"2024-05-09T08:56:00.696633+0000","last_active":"2024-05-09T08:58:32.500573+0000","last_peered":"2024-05-09T08:58:32.500573+0000","last_clean":"2024-05-09T08:58:32.500573+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:32.500573+0000","last_undegraded":"2024-05-09T08:58:32.500573+0000","last_fullsized":"2024-05-09T08:58:32.500573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:28.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"83'4","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.606250+0000","last_change":"2024-05-09T08:58:14.268728+0000","last_active":"2024-05-09T08:58:35.606250+0000","last_peered":"2024-05-09T08:58:35.606250+0000","last_clean":"2024-05-09T08:58:35.606250+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:35.606250+0000","last_undegraded":"2024-05-09T08:58:35.606250+0000","last_fullsized":"2024-05-09T08:58:35.606250+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:57:36.418440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"63'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395791+0000","last_change":"2024-05-09T08:56:00.693963+0000","last_active":"2024-05-09T08:58:30.395791+0000","last_peered":"2024-05-09T08:58:30.395791+0000","last_clean":"2024-05-09T08:58:30.395791+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:30.395791+0000","last_undegraded":"2024-05-09T08:58:30.395791+0000","last_fullsized":"2024-05-09T08:58:30.395791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:45:50.760589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"83'3","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:33.217416+0000","last_change":"2024-05-09T08:56:01.687188+0000","last_active":"2024-05-09T08:58:33.217416+0000","last_peered":"2024-05-09T08:58:33.217416+0000","last_clean":"2024-05-09T08:58:33.217416+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:33.217416+0000","last_undegraded":"2024-05-09T08:58:33.217416+0000","last_fullsized":"2024-05-09T08:58:33.217416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:36:04.021627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:35.618711+0000","last_change":"2024-05-09T08:58:13.294341+0000","last_active":"2024-05-09T08:58:35.618711+0000","last_peered":"2024-05-09T08:58:35.618711+0000","last_clean":"2024-05-09T08:58:35.618711+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:35.618711+0000","last_undegraded":"2024-05-09T08:58:35.618711+0000","last_fullsized":"2024-05-09T08:58:35.618711+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:33.387737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"20'5","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.394639+0000","last_change":"2024-05-09T08:56:00.697044+0000","last_active":"2024-05-09T08:58:30.394639+0000","last_peered":"2024-05-09T08:58:30.394639+0000","last_clean":"2024-05-09T08:58:30.394639+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:30.394639+0000","last_undegraded":"2024-05-09T08:58:30.394639+0000","last_fullsized":"2024-05-09T08:58:30.394639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:45.508945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":98,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395602+0000","last_change":"2024-05-09T08:58:24.726376+0000","last_active":"2024-05-09T08:58:30.395602+0000","last_peered":"2024-05-09T08:58:30.395602+0000","last_clean":"2024-05-09T08:58:30.395602+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:30.395602+0000","last_undegraded":"2024-05-09T08:58:30.395602+0000","last_fullsized":"2024-05-09T08:58:30.395602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:00.874805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"63'13","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.395948+0000","last_change":"2024-05-09T08:56:00.696884+0000","last_active":"2024-05-09T08:58:30.395948+0000","last_peered":"2024-05-09T08:58:30.395948+0000","last_clean":"2024-05-09T08:58:30.395948+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:30.395948+0000","last_undegraded":"2024-05-09T08:58:30.395948+0000","last_fullsized":"2024-05-09T08:58:30.395948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:18.288924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393681+0000","last_change":"2024-05-09T08:58:14.269138+0000","last_active":"2024-05-09T08:58:30.393681+0000","last_peered":"2024-05-09T08:58:30.393681+0000","last_clean":"2024-05-09T08:58:30.393681+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:30.393681+0000","last_undegraded":"2024-05-09T08:58:30.393681+0000","last_fullsized":"2024-05-09T08:58:30.393681+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:44:45.133788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397344+0000","last_change":"2024-05-09T08:58:14.272076+0000","last_active":"2024-05-09T08:58:30.397344+0000","last_peered":"2024-05-09T08:58:30.397344+0000","last_clean":"2024-05-09T08:58:30.397344+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:30.397344+0000","last_undegraded":"2024-05-09T08:58:30.397344+0000","last_fullsized":"2024-05-09T08:58:30.397344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:40.512741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":136,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.397299+0000","last_change":"2024-05-09T08:56:01.701687+0000","last_active":"2024-05-09T08:58:30.397299+0000","last_peered":"2024-05-09T08:58:30.397299+0000","last_clean":"2024-05-09T08:58:30.397299+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:30.397299+0000","last_undegraded":"2024-05-09T08:58:30.397299+0000","last_fullsized":"2024-05-09T08:58:30.397299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:56:00.667962+0000","last_clean_scrub_stamp":"2024-05-09T08:56:00.667962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:00:23.092682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'13","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:30.393664+0000","last_change":"2024-05-09T08:58:14.290847+0000","last_active":"2024-05-09T08:58:30.393664+0000","last_peered":"2024-05-09T08:58:30.393664+0000","last_clean":"2024-05-09T08:58:30.393664+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:30.393664+0000","last_undegraded":"2024-05-09T08:58:30.393664+0000","last_fullsized":"2024-05-09T08:58:30.393664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T08:55:59.663128+0000","last_clean_scrub_stamp":"2024-05-09T08:55:59.663128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:16:43.257826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44595,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":89190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343248,"statfs":{"total":96636764160,"available":96607485952,"internally_reserved":0,"allocated":1609728,"data_stored":1050594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1040384,"data_stored":495960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":461802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":459200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":496553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874247,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29800,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342040,"statfs":{"total":96636764160,"available":96606248960,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":459402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339096,"statfs":{"total":96636764160,"available":96603234304,"internally_reserved":0,"allocated":1601536,"data_stored":1049112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":38997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":38263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:58:42.355 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.356 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.356 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.356 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.356 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.357 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.357 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.357 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.357 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.357 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.358 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.358 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.358 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.358 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.359 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.359 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.359 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.359 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.360 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.360 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.360 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.360 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.361 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.361 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.361 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.361 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.362 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.362 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.362 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.362 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.363 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.363 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.363 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.363 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.363 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.364 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.365 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.366 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T08:55:56.645068+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=56, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.367 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.368 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.369 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.370 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.371 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.371 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.371 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-09T08:56:00.667962+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=56, tm_sec=0, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.371 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-05-09T08:55:59.663128+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=55, tm_sec=59, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=8, tm_min=58, tm_sec=37, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T08:58:42.371 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T08:59:02.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T08:59:02.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.490+0000 7f85c0c15640 1 Processor -- start 2024-05-09T08:59:02.490 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.490+0000 7f85c0c15640 1 -- start start 2024-05-09T08:59:02.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85c0c15640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85c0c15640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f85bc0c8780 0x7f85bc1622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85c0c15640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85bc0c8350 con 0x7f85bc0c9c00 2024-05-09T08:59:02.491 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85c0c15640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f85bc1521e0 con 0x7f85bc0c8780 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85c0c15640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc158490 con 0x7f85bc151e00 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85b1d9b640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f85bc0c9c00 0x7f85bc0c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:59208/0 (socket says 172.21.15.49:59208) 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85b1d9b640 1 -- 172.21.15.49:0/3001761174 learned_addr learned my addr 172.21.15.49:0/3001761174 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3001761174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.491+0000 7f85bad9d640 1 --2- 172.21.15.49:0/3001761174 >> v2:172.21.15.183:3300/0 conn(0x7f85bc0c8780 0x7f85bc1622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.492+0000 7f85ba59c640 1 -- 172.21.15.49:0/3001761174 >> v2:172.21.15.183:3300/0 conn(0x7f85bc0c8780 msgr2=0x7f85bc1622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.492+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3001761174 >> v2:172.21.15.183:3300/0 conn(0x7f85bc0c8780 0x7f85bc1622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.492+0000 7f85ba59c640 1 -- 172.21.15.49:0/3001761174 >> v1:172.21.15.49:6789/0 conn(0x7f85bc0c9c00 legacy=0x7f85bc0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.492+0000 7f85ba59c640 1 -- 172.21.15.49:0/3001761174 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc158750 con 0x7f85bc151e00 2024-05-09T08:59:02.492 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.493+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3001761174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc0c6e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0097a0 tx=0x7f85ac02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aeca235ff6934f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.494+0000 7f85b9d9b640 1 -- 172.21.15.49:0/3001761174 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac037070 con 0x7f85bc151e00 2024-05-09T08:59:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.494+0000 7f85b9d9b640 1 -- 172.21.15.49:0/3001761174 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ac032440 con 0x7f85bc151e00 2024-05-09T08:59:02.493 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.494+0000 7f85b9d9b640 1 -- 172.21.15.49:0/3001761174 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac032810 con 0x7f85bc151e00 2024-05-09T08:59:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 -- 172.21.15.49:0/3001761174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 msgr2=0x7f85bc0c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.494 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3001761174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc0c6e00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0097a0 tx=0x7f85ac02a210 comp rx=0 tx=0).stop 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 -- 172.21.15.49:0/3001761174 shutdown_connections 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3001761174 >> v2:172.21.15.183:3300/0 conn(0x7f85bc0c8780 0x7f85bc1622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3001761174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc0c6e00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 -- 172.21.15.49:0/3001761174 >> 172.21.15.49:0/3001761174 conn(0x7f85bc074840 msgr2=0x7f85bc074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 -- 172.21.15.49:0/3001761174 shutdown_connections 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.495+0000 7f85c0c15640 1 -- 172.21.15.49:0/3001761174 wait complete. 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.496+0000 7f85c0c15640 1 Processor -- start 2024-05-09T08:59:02.495 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 -- start start 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85bc1464a0 0x7f85bc15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc071bb0 con 0x7f85bc0c8780 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc0718d0 con 0x7f85bc1464a0 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85c0c15640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc071e90 con 0x7f85bc151e00 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.498 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85ba59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:47518/0 (socket says 172.21.15.49:47518) 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85ba59c640 1 -- 172.21.15.49:0/3219462967 learned_addr learned my addr 172.21.15.49:0/3219462967 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85bad9d640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85bc1464a0 0x7f85bc15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.497+0000 7f85b1d9b640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 -- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 msgr2=0x7f85bc156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 -- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85bc1464a0 msgr2=0x7f85bc15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85bc1464a0 0x7f85bc15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 -- 172.21.15.49:0/3219462967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc0c58b0 con 0x7f85bc0c8780 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85ba59c640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f85ac02a250 tx=0x7f85ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac041020 con 0x7f85bc0c8780 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ac030700 con 0x7f85bc0c8780 2024-05-09T08:59:02.499 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac037040 con 0x7f85bc0c8780 2024-05-09T08:59:02.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc163970 con 0x7f85bc0c8780 2024-05-09T08:59:02.500 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.498+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc163f30 con 0x7f85bc0c8780 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.499+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc068560 con 0x7f85bc0c8780 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.500+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85ac044020 con 0x7f85bc0c8780 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.500+0000 7f85b37fe640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 0x7f859c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.500+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f85ac0ab570 con 0x7f85bc0c8780 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.501+0000 7f85b1d9b640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 0x7f859c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.501 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.501+0000 7f85b1d9b640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 0x7f859c06e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0c7bb0 tx=0x7f85a4007e80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.504 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.505+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f85ac04b030 con 0x7f85bc0c8780 2024-05-09T08:59:02.685 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.685+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 --> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85bc0727e0 con 0x7f859c06be60 2024-05-09T08:59:02.691 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.692+0000 7f85b37fe640 1 -- 172.21.15.49:0/3219462967 <== mgr.4099 v2:172.21.15.183:6832/3271117140 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252719 (secure 0 0 0) 0x7f85bc0727e0 con 0x7f859c06be60 2024-05-09T08:59:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:59:02.695 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-05-09T08:59:02.696 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 msgr2=0x7f859c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 0x7f859c06e320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0c7bb0 tx=0x7f85a4007e80 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 msgr2=0x7f85bc1f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f85ac02a250 tx=0x7f85ac0046c0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 shutdown_connections 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f859c06be60 0x7f859c06e320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85bc1464a0 0x7f85bc15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f85bc151e00 0x7f85bc156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 --2- 172.21.15.49:0/3219462967 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f85bc0c8780 0x7f85bc1f1510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 >> 172.21.15.49:0/3219462967 conn(0x7f85bc074840 msgr2=0x7f85bc070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 shutdown_connections 2024-05-09T08:59:02.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.695+0000 7f85c0c15640 1 -- 172.21.15.49:0/3219462967 wait complete. 2024-05-09T08:59:02.706 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-05-09T08:59:02.610885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":635039,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":415,"num_write_kb":2541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1182051,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243704,"kb_used_data":9144,"kb_used_omap":24,"kb_used_meta":234471,"kb_avail":754731016,"statfs":{"total":773094113280,"available":772844560384,"internally_reserved":0,"allocated":9363456,"data_stored":4933038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25528,"internal_metadata":240098376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003181"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.050349+0000","last_change":"2024-05-09T08:58:42.050404+0000","last_active":"2024-05-09T08:58:42.050349+0000","last_peered":"2024-05-09T08:58:42.050349+0000","last_clean":"2024-05-09T08:58:42.050349+0000","last_became_active":"2024-05-09T08:56:01.699497+0000","last_became_peered":"2024-05-09T08:56:01.699497+0000","last_unstale":"2024-05-09T08:58:42.050349+0000","last_undegraded":"2024-05-09T08:58:42.050349+0000","last_fullsized":"2024-05-09T08:58:42.050349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:42.050230+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:42.050230+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.050230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:55:58.937353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":160,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:51.153820+0000","last_change":"2024-05-09T08:58:51.153856+0000","last_active":"2024-05-09T08:58:51.153820+0000","last_peered":"2024-05-09T08:58:51.153820+0000","last_clean":"2024-05-09T08:58:51.153820+0000","last_became_active":"2024-05-09T08:58:14.269585+0000","last_became_peered":"2024-05-09T08:58:14.269585+0000","last_unstale":"2024-05-09T08:58:51.153820+0000","last_undegraded":"2024-05-09T08:58:51.153820+0000","last_fullsized":"2024-05-09T08:58:51.153820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:51.153728+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:51.153728+0000","last_clean_scrub_stamp":"2024-05-09T08:58:51.153728+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:48.345714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.110797+0000","last_change":"2024-05-09T08:58:41.110819+0000","last_active":"2024-05-09T08:58:41.110797+0000","last_peered":"2024-05-09T08:58:41.110797+0000","last_clean":"2024-05-09T08:58:41.110797+0000","last_became_active":"2024-05-09T08:58:14.263331+0000","last_became_peered":"2024-05-09T08:58:14.263331+0000","last_unstale":"2024-05-09T08:58:41.110797+0000","last_undegraded":"2024-05-09T08:58:41.110797+0000","last_fullsized":"2024-05-09T08:58:41.110797+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:41.110733+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:41.110733+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.110733+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:42:00.858980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.079305+0000","last_change":"2024-05-09T08:58:41.079330+0000","last_active":"2024-05-09T08:58:41.079305+0000","last_peered":"2024-05-09T08:58:41.079305+0000","last_clean":"2024-05-09T08:58:41.079305+0000","last_became_active":"2024-05-09T08:58:13.286041+0000","last_became_peered":"2024-05-09T08:58:13.286041+0000","last_unstale":"2024-05-09T08:58:41.079305+0000","last_undegraded":"2024-05-09T08:58:41.079305+0000","last_fullsized":"2024-05-09T08:58:41.079305+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:41.079231+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:41.079231+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.079231+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:02.067463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:54.974392+0000","last_change":"2024-05-09T08:58:54.974421+0000","last_active":"2024-05-09T08:58:54.974392+0000","last_peered":"2024-05-09T08:58:54.974392+0000","last_clean":"2024-05-09T08:58:54.974392+0000","last_became_active":"2024-05-09T08:58:13.296657+0000","last_became_peered":"2024-05-09T08:58:13.296657+0000","last_unstale":"2024-05-09T08:58:54.974392+0000","last_undegraded":"2024-05-09T08:58:54.974392+0000","last_fullsized":"2024-05-09T08:58:54.974392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:54.974269+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:54.974269+0000","last_clean_scrub_stamp":"2024-05-09T08:58:54.974269+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:29.659977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":113,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:54.058181+0000","last_change":"2024-05-09T08:58:54.058210+0000","last_active":"2024-05-09T08:58:54.058181+0000","last_peered":"2024-05-09T08:58:54.058181+0000","last_clean":"2024-05-09T08:58:54.058181+0000","last_became_active":"2024-05-09T08:58:24.725796+0000","last_became_peered":"2024-05-09T08:58:24.725796+0000","last_unstale":"2024-05-09T08:58:54.058181+0000","last_undegraded":"2024-05-09T08:58:54.058181+0000","last_fullsized":"2024-05-09T08:58:54.058181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:54.058090+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:54.058090+0000","last_clean_scrub_stamp":"2024-05-09T08:58:54.058090+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:14.350721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":66,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.066161+0000","last_change":"2024-05-09T08:58:47.066202+0000","last_active":"2024-05-09T08:58:47.066161+0000","last_peered":"2024-05-09T08:58:47.066161+0000","last_clean":"2024-05-09T08:58:47.066161+0000","last_became_active":"2024-05-09T08:58:14.263767+0000","last_became_peered":"2024-05-09T08:58:14.263767+0000","last_unstale":"2024-05-09T08:58:47.066161+0000","last_undegraded":"2024-05-09T08:58:47.066161+0000","last_fullsized":"2024-05-09T08:58:47.066161+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:47.066062+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:47.066062+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.066062+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:55:26.879179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.278880+0000","last_change":"2024-05-09T08:58:46.278905+0000","last_active":"2024-05-09T08:58:46.278880+0000","last_peered":"2024-05-09T08:58:46.278880+0000","last_clean":"2024-05-09T08:58:46.278880+0000","last_became_active":"2024-05-09T08:56:01.699727+0000","last_became_peered":"2024-05-09T08:56:01.699727+0000","last_unstale":"2024-05-09T08:58:46.278880+0000","last_undegraded":"2024-05-09T08:58:46.278880+0000","last_fullsized":"2024-05-09T08:58:46.278880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:46.278792+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:46.278792+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.278792+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:35:30.720598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.962925+0000","last_change":"2024-05-09T08:58:50.962957+0000","last_active":"2024-05-09T08:58:50.962925+0000","last_peered":"2024-05-09T08:58:50.962925+0000","last_clean":"2024-05-09T08:58:50.962925+0000","last_became_active":"2024-05-09T08:56:01.700210+0000","last_became_peered":"2024-05-09T08:56:01.700210+0000","last_unstale":"2024-05-09T08:58:50.962925+0000","last_undegraded":"2024-05-09T08:58:50.962925+0000","last_fullsized":"2024-05-09T08:58:50.962925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:50.962810+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:50.962810+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.962810+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:20:50.023271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":160,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.520201+0000","last_change":"2024-05-09T08:58:50.520255+0000","last_active":"2024-05-09T08:58:50.520201+0000","last_peered":"2024-05-09T08:58:50.520201+0000","last_clean":"2024-05-09T08:58:50.520201+0000","last_became_active":"2024-05-09T08:56:01.693985+0000","last_became_peered":"2024-05-09T08:56:01.693985+0000","last_unstale":"2024-05-09T08:58:50.520201+0000","last_undegraded":"2024-05-09T08:58:50.520201+0000","last_fullsized":"2024-05-09T08:58:50.520201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'16","last_scrub_stamp":"2024-05-09T08:58:50.520085+0000","last_deep_scrub":"83'16","last_deep_scrub_stamp":"2024-05-09T08:58:50.520085+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.520085+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:17.548047+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:49.484418+0000","last_change":"2024-05-09T08:58:49.484467+0000","last_active":"2024-05-09T08:58:49.484418+0000","last_peered":"2024-05-09T08:58:49.484418+0000","last_clean":"2024-05-09T08:58:49.484418+0000","last_became_active":"2024-05-09T08:56:01.693798+0000","last_became_peered":"2024-05-09T08:56:01.693798+0000","last_unstale":"2024-05-09T08:58:49.484418+0000","last_undegraded":"2024-05-09T08:58:49.484418+0000","last_fullsized":"2024-05-09T08:58:49.484418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:49.484298+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:49.484298+0000","last_clean_scrub_stamp":"2024-05-09T08:58:49.484298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:29:52.199640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.904973+0000","last_change":"2024-05-09T08:58:42.905015+0000","last_active":"2024-05-09T08:58:42.904973+0000","last_peered":"2024-05-09T08:58:42.904973+0000","last_clean":"2024-05-09T08:58:42.904973+0000","last_became_active":"2024-05-09T08:56:01.697954+0000","last_became_peered":"2024-05-09T08:56:01.697954+0000","last_unstale":"2024-05-09T08:58:42.904973+0000","last_undegraded":"2024-05-09T08:58:42.904973+0000","last_fullsized":"2024-05-09T08:58:42.904973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:42.904861+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:42.904861+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.904861+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:45:50.597067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.304679+0000","last_change":"2024-05-09T08:58:48.304728+0000","last_active":"2024-05-09T08:58:48.304679+0000","last_peered":"2024-05-09T08:58:48.304679+0000","last_clean":"2024-05-09T08:58:48.304679+0000","last_became_active":"2024-05-09T08:56:01.700530+0000","last_became_peered":"2024-05-09T08:56:01.700530+0000","last_unstale":"2024-05-09T08:58:48.304679+0000","last_undegraded":"2024-05-09T08:58:48.304679+0000","last_fullsized":"2024-05-09T08:58:48.304679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:48.304549+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:48.304549+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.304549+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:56.416515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"83'10","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.075817+0000","last_change":"2024-05-09T08:58:50.075851+0000","last_active":"2024-05-09T08:58:50.075817+0000","last_peered":"2024-05-09T08:58:50.075817+0000","last_clean":"2024-05-09T08:58:50.075817+0000","last_became_active":"2024-05-09T08:58:14.266037+0000","last_became_peered":"2024-05-09T08:58:14.266037+0000","last_unstale":"2024-05-09T08:58:50.075817+0000","last_undegraded":"2024-05-09T08:58:50.075817+0000","last_fullsized":"2024-05-09T08:58:50.075817+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:50.075721+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:50.075721+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.075721+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:59:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.123397+0000","last_change":"2024-05-09T08:58:45.123426+0000","last_active":"2024-05-09T08:58:45.123397+0000","last_peered":"2024-05-09T08:58:45.123397+0000","last_clean":"2024-05-09T08:58:45.123397+0000","last_became_active":"2024-05-09T08:56:01.687279+0000","last_became_peered":"2024-05-09T08:56:01.687279+0000","last_unstale":"2024-05-09T08:58:45.123397+0000","last_undegraded":"2024-05-09T08:58:45.123397+0000","last_fullsized":"2024-05-09T08:58:45.123397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:45.123272+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:45.123272+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.123272+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:31:08.973020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":116,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:53.088792+0000","last_change":"2024-05-09T08:58:53.088826+0000","last_active":"2024-05-09T08:58:53.088792+0000","last_peered":"2024-05-09T08:58:53.088792+0000","last_clean":"2024-05-09T08:58:53.088792+0000","last_became_active":"2024-05-09T08:58:24.725463+0000","last_became_peered":"2024-05-09T08:58:24.725463+0000","last_unstale":"2024-05-09T08:58:53.088792+0000","last_undegraded":"2024-05-09T08:58:53.088792+0000","last_fullsized":"2024-05-09T08:58:53.088792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:53.088697+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:53.088697+0000","last_clean_scrub_stamp":"2024-05-09T08:58:53.088697+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:07:37.788961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.060557+0000","last_change":"2024-05-09T08:58:47.060607+0000","last_active":"2024-05-09T08:58:47.060557+0000","last_peered":"2024-05-09T08:58:47.060557+0000","last_clean":"2024-05-09T08:58:47.060557+0000","last_became_active":"2024-05-09T08:56:01.699092+0000","last_became_peered":"2024-05-09T08:56:01.699092+0000","last_unstale":"2024-05-09T08:58:47.060557+0000","last_undegraded":"2024-05-09T08:58:47.060557+0000","last_fullsized":"2024-05-09T08:58:47.060557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:47.060446+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:47.060446+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.060446+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:28.768840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":72,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.029695+0000","last_change":"2024-05-09T08:58:44.029740+0000","last_active":"2024-05-09T08:58:44.029695+0000","last_peered":"2024-05-09T08:58:44.029695+0000","last_clean":"2024-05-09T08:58:44.029695+0000","last_became_active":"2024-05-09T08:58:13.290575+0000","last_became_peered":"2024-05-09T08:58:13.290575+0000","last_unstale":"2024-05-09T08:58:44.029695+0000","last_undegraded":"2024-05-09T08:58:44.029695+0000","last_fullsized":"2024-05-09T08:58:44.029695+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:44.029600+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:44.029600+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.029600+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:52.053930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:53.951199+0000","last_change":"2024-05-09T08:58:53.951273+0000","last_active":"2024-05-09T08:58:53.951199+0000","last_peered":"2024-05-09T08:58:53.951199+0000","last_clean":"2024-05-09T08:58:53.951199+0000","last_became_active":"2024-05-09T08:58:13.286823+0000","last_became_peered":"2024-05-09T08:58:13.286823+0000","last_unstale":"2024-05-09T08:58:53.951199+0000","last_undegraded":"2024-05-09T08:58:53.951199+0000","last_fullsized":"2024-05-09T08:58:53.951199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:53.951099+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:53.951099+0000","last_clean_scrub_stamp":"2024-05-09T08:58:53.951099+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:16:44.193303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.512931+0000","last_change":"2024-05-09T08:58:48.512958+0000","last_active":"2024-05-09T08:58:48.512931+0000","last_peered":"2024-05-09T08:58:48.512931+0000","last_clean":"2024-05-09T08:58:48.512931+0000","last_became_active":"2024-05-09T08:56:01.688668+0000","last_became_peered":"2024-05-09T08:56:01.688668+0000","last_unstale":"2024-05-09T08:58:48.512931+0000","last_undegraded":"2024-05-09T08:58:48.512931+0000","last_fullsized":"2024-05-09T08:58:48.512931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:48.512836+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:48.512836+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.512836+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:00.073261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":62,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.122546+0000","last_change":"2024-05-09T08:58:46.122606+0000","last_active":"2024-05-09T08:58:46.122546+0000","last_peered":"2024-05-09T08:58:46.122546+0000","last_clean":"2024-05-09T08:58:46.122546+0000","last_became_active":"2024-05-09T08:58:14.267249+0000","last_became_peered":"2024-05-09T08:58:14.267249+0000","last_unstale":"2024-05-09T08:58:46.122546+0000","last_undegraded":"2024-05-09T08:58:46.122546+0000","last_fullsized":"2024-05-09T08:58:46.122546+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:46.122456+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:46.122456+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.122456+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:05:04.983988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.027213+0000","last_change":"2024-05-09T08:58:48.027261+0000","last_active":"2024-05-09T08:58:48.027213+0000","last_peered":"2024-05-09T08:58:48.027213+0000","last_clean":"2024-05-09T08:58:48.027213+0000","last_became_active":"2024-05-09T08:56:01.699126+0000","last_became_peered":"2024-05-09T08:56:01.699126+0000","last_unstale":"2024-05-09T08:58:48.027213+0000","last_undegraded":"2024-05-09T08:58:48.027213+0000","last_fullsized":"2024-05-09T08:58:48.027213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:48.027107+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:48.027107+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.027107+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:16.150833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.263479+0000","last_change":"2024-05-09T08:58:44.263506+0000","last_active":"2024-05-09T08:58:44.263479+0000","last_peered":"2024-05-09T08:58:44.263479+0000","last_clean":"2024-05-09T08:58:44.263479+0000","last_became_active":"2024-05-09T08:56:01.694343+0000","last_became_peered":"2024-05-09T08:56:01.694343+0000","last_unstale":"2024-05-09T08:58:44.263479+0000","last_undegraded":"2024-05-09T08:58:44.263479+0000","last_fullsized":"2024-05-09T08:58:44.263479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:44.263376+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:44.263376+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.263376+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:13:14.390095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.994047+0000","last_change":"2024-05-09T08:58:48.994104+0000","last_active":"2024-05-09T08:58:48.994047+0000","last_peered":"2024-05-09T08:58:48.994047+0000","last_clean":"2024-05-09T08:58:48.994047+0000","last_became_active":"2024-05-09T08:56:01.700415+0000","last_became_peered":"2024-05-09T08:56:01.700415+0000","last_unstale":"2024-05-09T08:58:48.994047+0000","last_undegraded":"2024-05-09T08:58:48.994047+0000","last_fullsized":"2024-05-09T08:58:48.994047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:48.993939+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:48.993939+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.993939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:09:27.818701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.299526+0000","last_change":"2024-05-09T08:58:43.299558+0000","last_active":"2024-05-09T08:58:43.299526+0000","last_peered":"2024-05-09T08:58:43.299526+0000","last_clean":"2024-05-09T08:58:43.299526+0000","last_became_active":"2024-05-09T08:56:01.693458+0000","last_became_peered":"2024-05-09T08:56:01.693458+0000","last_unstale":"2024-05-09T08:58:43.299526+0000","last_undegraded":"2024-05-09T08:58:43.299526+0000","last_fullsized":"2024-05-09T08:58:43.299526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'2","last_scrub_stamp":"2024-05-09T08:58:43.299415+0000","last_deep_scrub":"83'2","last_deep_scrub_stamp":"2024-05-09T08:58:43.299415+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.299415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:41:31.678872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:51.512896+0000","last_change":"2024-05-09T08:58:51.512924+0000","last_active":"2024-05-09T08:58:51.512896+0000","last_peered":"2024-05-09T08:58:51.512896+0000","last_clean":"2024-05-09T08:58:51.512896+0000","last_became_active":"2024-05-09T08:56:01.697182+0000","last_became_peered":"2024-05-09T08:56:01.697182+0000","last_unstale":"2024-05-09T08:58:51.512896+0000","last_undegraded":"2024-05-09T08:58:51.512896+0000","last_fullsized":"2024-05-09T08:58:51.512896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-05-09T08:58:51.512783+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-05-09T08:58:51.512783+0000","last_clean_scrub_stamp":"2024-05-09T08:58:51.512783+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:18:43.714327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.104478+0000","last_change":"2024-05-09T08:58:46.104503+0000","last_active":"2024-05-09T08:58:46.104478+0000","last_peered":"2024-05-09T08:58:46.104478+0000","last_clean":"2024-05-09T08:58:46.104478+0000","last_became_active":"2024-05-09T08:56:01.693955+0000","last_became_peered":"2024-05-09T08:56:01.693955+0000","last_unstale":"2024-05-09T08:58:46.104478+0000","last_undegraded":"2024-05-09T08:58:46.104478+0000","last_fullsized":"2024-05-09T08:58:46.104478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:46.104398+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:46.104398+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.104398+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:47.606438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.102763+0000","last_change":"2024-05-09T08:58:48.102763+0000","last_active":"2024-05-09T08:58:48.102763+0000","last_peered":"2024-05-09T08:58:48.102763+0000","last_clean":"2024-05-09T08:58:48.102763+0000","last_became_active":"2024-05-09T08:56:50.303226+0000","last_became_peered":"2024-05-09T08:56:50.303226+0000","last_unstale":"2024-05-09T08:58:48.102763+0000","last_undegraded":"2024-05-09T08:58:48.102763+0000","last_fullsized":"2024-05-09T08:58:48.102763+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:48.102646+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:48.102646+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.102646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:23.942303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.089365+0000","last_change":"2024-05-09T08:58:42.089387+0000","last_active":"2024-05-09T08:58:42.089365+0000","last_peered":"2024-05-09T08:58:42.089365+0000","last_clean":"2024-05-09T08:58:42.089365+0000","last_became_active":"2024-05-09T08:58:14.264723+0000","last_became_peered":"2024-05-09T08:58:14.264723+0000","last_unstale":"2024-05-09T08:58:42.089365+0000","last_undegraded":"2024-05-09T08:58:42.089365+0000","last_fullsized":"2024-05-09T08:58:42.089365+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:42.089293+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:42.089293+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.089293+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:17:46.081828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:52.043150+0000","last_change":"2024-05-09T08:58:52.043183+0000","last_active":"2024-05-09T08:58:52.043150+0000","last_peered":"2024-05-09T08:58:52.043150+0000","last_clean":"2024-05-09T08:58:52.043150+0000","last_became_active":"2024-05-09T08:58:24.616468+0000","last_became_peered":"2024-05-09T08:58:24.616468+0000","last_unstale":"2024-05-09T08:58:52.043150+0000","last_undegraded":"2024-05-09T08:58:52.043150+0000","last_fullsized":"2024-05-09T08:58:52.043150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:52.043045+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:52.043045+0000","last_clean_scrub_stamp":"2024-05-09T08:58:52.043045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:46:11.458075+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":74,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.077927+0000","last_change":"2024-05-09T08:58:43.078003+0000","last_active":"2024-05-09T08:58:43.077927+0000","last_peered":"2024-05-09T08:58:43.077927+0000","last_clean":"2024-05-09T08:58:43.077927+0000","last_became_active":"2024-05-09T08:58:13.288754+0000","last_became_peered":"2024-05-09T08:58:13.288754+0000","last_unstale":"2024-05-09T08:58:43.077927+0000","last_undegraded":"2024-05-09T08:58:43.077927+0000","last_fullsized":"2024-05-09T08:58:43.077927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'12","last_scrub_stamp":"2024-05-09T08:58:43.077801+0000","last_deep_scrub":"83'12","last_deep_scrub_stamp":"2024-05-09T08:58:43.077801+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.077801+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:21.071683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":163,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:52.532963+0000","last_change":"2024-05-09T08:58:52.532990+0000","last_active":"2024-05-09T08:58:52.532963+0000","last_peered":"2024-05-09T08:58:52.532963+0000","last_clean":"2024-05-09T08:58:52.532963+0000","last_became_active":"2024-05-09T08:58:14.271662+0000","last_became_peered":"2024-05-09T08:58:14.271662+0000","last_unstale":"2024-05-09T08:58:52.532963+0000","last_undegraded":"2024-05-09T08:58:52.532963+0000","last_fullsized":"2024-05-09T08:58:52.532963+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:52.532868+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:52.532868+0000","last_clean_scrub_stamp":"2024-05-09T08:58:52.532868+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:12:07.126428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:40.937886+0000","last_change":"2024-05-09T08:58:40.937915+0000","last_active":"2024-05-09T08:58:40.937886+0000","last_peered":"2024-05-09T08:58:40.937886+0000","last_clean":"2024-05-09T08:58:40.937886+0000","last_became_active":"2024-05-09T08:56:01.688722+0000","last_became_peered":"2024-05-09T08:56:01.688722+0000","last_unstale":"2024-05-09T08:58:40.937886+0000","last_undegraded":"2024-05-09T08:58:40.937886+0000","last_fullsized":"2024-05-09T08:58:40.937886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:40.937780+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:40.937780+0000","last_clean_scrub_stamp":"2024-05-09T08:58:40.937780+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:58:53.310038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.223080+0000","last_change":"2024-05-09T08:58:41.223107+0000","last_active":"2024-05-09T08:58:41.223080+0000","last_peered":"2024-05-09T08:58:41.223080+0000","last_clean":"2024-05-09T08:58:41.223080+0000","last_became_active":"2024-05-09T08:56:00.693826+0000","last_became_peered":"2024-05-09T08:56:00.693826+0000","last_unstale":"2024-05-09T08:58:41.223080+0000","last_undegraded":"2024-05-09T08:58:41.223080+0000","last_fullsized":"2024-05-09T08:58:41.223080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:41.223003+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:41.223003+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.223003+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:45:43.934060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:40.148197+0000","last_change":"2024-05-09T08:58:40.148246+0000","last_active":"2024-05-09T08:58:40.148197+0000","last_peered":"2024-05-09T08:58:40.148197+0000","last_clean":"2024-05-09T08:58:40.148197+0000","last_became_active":"2024-05-09T08:58:14.261333+0000","last_became_peered":"2024-05-09T08:58:14.261333+0000","last_unstale":"2024-05-09T08:58:40.148197+0000","last_undegraded":"2024-05-09T08:58:40.148197+0000","last_fullsized":"2024-05-09T08:58:40.148197+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:40.148100+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:40.148100+0000","last_clean_scrub_stamp":"2024-05-09T08:58:40.148100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:59:15.682960+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"83'10","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.484372+0000","last_change":"2024-05-09T08:58:42.484450+0000","last_active":"2024-05-09T08:58:42.484372+0000","last_peered":"2024-05-09T08:58:42.484372+0000","last_clean":"2024-05-09T08:58:42.484372+0000","last_became_active":"2024-05-09T08:56:00.687660+0000","last_became_peered":"2024-05-09T08:56:00.687660+0000","last_unstale":"2024-05-09T08:58:42.484372+0000","last_undegraded":"2024-05-09T08:58:42.484372+0000","last_fullsized":"2024-05-09T08:58:42.484372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:42.484264+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:42.484264+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.484264+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:13:43.910039+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"83'5","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:40.204250+0000","last_change":"2024-05-09T08:58:40.204275+0000","last_active":"2024-05-09T08:58:40.204250+0000","last_peered":"2024-05-09T08:58:40.204250+0000","last_clean":"2024-05-09T08:58:40.204250+0000","last_became_active":"2024-05-09T08:56:00.691759+0000","last_became_peered":"2024-05-09T08:56:00.691759+0000","last_unstale":"2024-05-09T08:58:40.204250+0000","last_undegraded":"2024-05-09T08:58:40.204250+0000","last_fullsized":"2024-05-09T08:58:40.204250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:40.204173+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:40.204173+0000","last_clean_scrub_stamp":"2024-05-09T08:58:40.204173+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:33:35.912634+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":163,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:51.298497+0000","last_change":"2024-05-09T08:58:51.298559+0000","last_active":"2024-05-09T08:58:51.298497+0000","last_peered":"2024-05-09T08:58:51.298497+0000","last_clean":"2024-05-09T08:58:51.298497+0000","last_became_active":"2024-05-09T08:58:14.271500+0000","last_became_peered":"2024-05-09T08:58:14.271500+0000","last_unstale":"2024-05-09T08:58:51.298497+0000","last_undegraded":"2024-05-09T08:58:51.298497+0000","last_fullsized":"2024-05-09T08:58:51.298497+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:51.298385+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:51.298385+0000","last_clean_scrub_stamp":"2024-05-09T08:58:51.298385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:59:52.160368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'4","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.065164+0000","last_change":"2024-05-09T08:58:41.065191+0000","last_active":"2024-05-09T08:58:41.065164+0000","last_peered":"2024-05-09T08:58:41.065164+0000","last_clean":"2024-05-09T08:58:41.065164+0000","last_became_active":"2024-05-09T08:56:00.682623+0000","last_became_peered":"2024-05-09T08:56:00.682623+0000","last_unstale":"2024-05-09T08:58:41.065164+0000","last_undegraded":"2024-05-09T08:58:41.065164+0000","last_fullsized":"2024-05-09T08:58:41.065164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:41.065083+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:41.065083+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.065083+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:41:08.555592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'10","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.131252+0000","last_change":"2024-05-09T08:58:47.131309+0000","last_active":"2024-05-09T08:58:47.131252+0000","last_peered":"2024-05-09T08:58:47.131252+0000","last_clean":"2024-05-09T08:58:47.131252+0000","last_became_active":"2024-05-09T08:56:01.696447+0000","last_became_peered":"2024-05-09T08:56:01.696447+0000","last_unstale":"2024-05-09T08:58:47.131252+0000","last_undegraded":"2024-05-09T08:58:47.131252+0000","last_fullsized":"2024-05-09T08:58:47.131252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:47.131142+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:47.131142+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.131142+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:44.953059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"83'9","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.526142+0000","last_change":"2024-05-09T08:58:44.526177+0000","last_active":"2024-05-09T08:58:44.526142+0000","last_peered":"2024-05-09T08:58:44.526142+0000","last_clean":"2024-05-09T08:58:44.526142+0000","last_became_active":"2024-05-09T08:56:00.689232+0000","last_became_peered":"2024-05-09T08:56:00.689232+0000","last_unstale":"2024-05-09T08:58:44.526142+0000","last_undegraded":"2024-05-09T08:58:44.526142+0000","last_fullsized":"2024-05-09T08:58:44.526142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:44.526047+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:44.526047+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.526047+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:31:57.846678+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.078424+0000","last_change":"2024-05-09T08:58:46.078453+0000","last_active":"2024-05-09T08:58:46.078424+0000","last_peered":"2024-05-09T08:58:46.078424+0000","last_clean":"2024-05-09T08:58:46.078424+0000","last_became_active":"2024-05-09T08:56:01.698489+0000","last_became_peered":"2024-05-09T08:56:01.698489+0000","last_unstale":"2024-05-09T08:58:46.078424+0000","last_undegraded":"2024-05-09T08:58:46.078424+0000","last_fullsized":"2024-05-09T08:58:46.078424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'10","last_scrub_stamp":"2024-05-09T08:58:46.078337+0000","last_deep_scrub":"83'10","last_deep_scrub_stamp":"2024-05-09T08:58:46.078337+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.078337+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:44:33.481965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"83'6","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.073389+0000","last_change":"2024-05-09T08:58:42.073422+0000","last_active":"2024-05-09T08:58:42.073389+0000","last_peered":"2024-05-09T08:58:42.073389+0000","last_clean":"2024-05-09T08:58:42.073389+0000","last_became_active":"2024-05-09T08:56:00.684410+0000","last_became_peered":"2024-05-09T08:56:00.684410+0000","last_unstale":"2024-05-09T08:58:42.073389+0000","last_undegraded":"2024-05-09T08:58:42.073389+0000","last_fullsized":"2024-05-09T08:58:42.073389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:42.073304+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:42.073304+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.073304+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:12.907414+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.093904+0000","last_change":"2024-05-09T08:58:45.093939+0000","last_active":"2024-05-09T08:58:45.093904+0000","last_peered":"2024-05-09T08:58:45.093904+0000","last_clean":"2024-05-09T08:58:45.093904+0000","last_became_active":"2024-05-09T08:58:14.267242+0000","last_became_peered":"2024-05-09T08:58:14.267242+0000","last_unstale":"2024-05-09T08:58:45.093904+0000","last_undegraded":"2024-05-09T08:58:45.093904+0000","last_fullsized":"2024-05-09T08:58:45.093904+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:45.093814+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:45.093814+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.093814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:01:50.539748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'11","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.099584+0000","last_change":"2024-05-09T08:58:41.099617+0000","last_active":"2024-05-09T08:58:41.099584+0000","last_peered":"2024-05-09T08:58:41.099584+0000","last_clean":"2024-05-09T08:58:41.099584+0000","last_became_active":"2024-05-09T08:56:00.690796+0000","last_became_peered":"2024-05-09T08:56:00.690796+0000","last_unstale":"2024-05-09T08:58:41.099584+0000","last_undegraded":"2024-05-09T08:58:41.099584+0000","last_fullsized":"2024-05-09T08:58:41.099584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:41.099510+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:41.099510+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.099510+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:11:44.482551+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:41.936036+0000","last_change":"2024-05-09T08:58:41.936078+0000","last_active":"2024-05-09T08:58:41.936036+0000","last_peered":"2024-05-09T08:58:41.936036+0000","last_clean":"2024-05-09T08:58:41.936036+0000","last_became_active":"2024-05-09T08:56:01.695437+0000","last_became_peered":"2024-05-09T08:56:01.695437+0000","last_unstale":"2024-05-09T08:58:41.936036+0000","last_undegraded":"2024-05-09T08:58:41.936036+0000","last_fullsized":"2024-05-09T08:58:41.936036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:41.935920+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:41.935920+0000","last_clean_scrub_stamp":"2024-05-09T08:58:41.935920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:28:17.660754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"83'33","reported_seq":179,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.560529+0000","last_change":"2024-05-09T08:58:46.560567+0000","last_active":"2024-05-09T08:58:46.560529+0000","last_peered":"2024-05-09T08:58:46.560529+0000","last_clean":"2024-05-09T08:58:46.560529+0000","last_became_active":"2024-05-09T08:56:00.696699+0000","last_became_peered":"2024-05-09T08:56:00.696699+0000","last_unstale":"2024-05-09T08:58:46.560529+0000","last_undegraded":"2024-05-09T08:58:46.560529+0000","last_fullsized":"2024-05-09T08:58:46.560529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'33","last_scrub_stamp":"2024-05-09T08:58:46.560441+0000","last_deep_scrub":"83'33","last_deep_scrub_stamp":"2024-05-09T08:58:46.560441+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.560441+0000","objects_scrubbed":6,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:36:13.225170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37549,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:49.007829+0000","last_change":"2024-05-09T08:58:49.007870+0000","last_active":"2024-05-09T08:58:49.007829+0000","last_peered":"2024-05-09T08:58:49.007829+0000","last_clean":"2024-05-09T08:58:49.007829+0000","last_became_active":"2024-05-09T08:56:01.697317+0000","last_became_peered":"2024-05-09T08:56:01.697317+0000","last_unstale":"2024-05-09T08:58:49.007829+0000","last_undegraded":"2024-05-09T08:58:49.007829+0000","last_fullsized":"2024-05-09T08:58:49.007829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:49.007743+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:49.007743+0000","last_clean_scrub_stamp":"2024-05-09T08:58:49.007743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:14:04.686216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"83'15","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.144375+0000","last_change":"2024-05-09T08:58:48.144416+0000","last_active":"2024-05-09T08:58:48.144375+0000","last_peered":"2024-05-09T08:58:48.144375+0000","last_clean":"2024-05-09T08:58:48.144375+0000","last_became_active":"2024-05-09T08:58:14.260884+0000","last_became_peered":"2024-05-09T08:58:14.260884+0000","last_unstale":"2024-05-09T08:58:48.144375+0000","last_undegraded":"2024-05-09T08:58:48.144375+0000","last_fullsized":"2024-05-09T08:58:48.144375+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-05-09T08:58:48.144291+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-05-09T08:58:48.144291+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.144291+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:06.638634+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":60,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.096552+0000","last_change":"2024-05-09T08:58:43.096600+0000","last_active":"2024-05-09T08:58:43.096552+0000","last_peered":"2024-05-09T08:58:43.096552+0000","last_clean":"2024-05-09T08:58:43.096552+0000","last_became_active":"2024-05-09T08:58:14.265181+0000","last_became_peered":"2024-05-09T08:58:14.265181+0000","last_unstale":"2024-05-09T08:58:43.096552+0000","last_undegraded":"2024-05-09T08:58:43.096552+0000","last_fullsized":"2024-05-09T08:58:43.096552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:43.096446+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:43.096446+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.096446+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:16:57.013639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"83'9","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.028599+0000","last_change":"2024-05-09T08:58:43.028635+0000","last_active":"2024-05-09T08:58:43.028599+0000","last_peered":"2024-05-09T08:58:43.028599+0000","last_clean":"2024-05-09T08:58:43.028599+0000","last_became_active":"2024-05-09T08:56:00.691712+0000","last_became_peered":"2024-05-09T08:56:00.691712+0000","last_unstale":"2024-05-09T08:58:43.028599+0000","last_undegraded":"2024-05-09T08:58:43.028599+0000","last_fullsized":"2024-05-09T08:58:43.028599+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:43.028516+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:43.028516+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.028516+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:17:06.105168+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":160,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.000571+0000","last_change":"2024-05-09T08:58:50.000595+0000","last_active":"2024-05-09T08:58:50.000571+0000","last_peered":"2024-05-09T08:58:50.000571+0000","last_clean":"2024-05-09T08:58:50.000571+0000","last_became_active":"2024-05-09T08:56:01.699358+0000","last_became_peered":"2024-05-09T08:56:01.699358+0000","last_unstale":"2024-05-09T08:58:50.000571+0000","last_undegraded":"2024-05-09T08:58:50.000571+0000","last_fullsized":"2024-05-09T08:58:50.000571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-05-09T08:58:50.000496+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-05-09T08:58:50.000496+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.000496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:39:11.520196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.492637+0000","last_change":"2024-05-09T08:58:43.492689+0000","last_active":"2024-05-09T08:58:43.492637+0000","last_peered":"2024-05-09T08:58:43.492637+0000","last_clean":"2024-05-09T08:58:43.492637+0000","last_became_active":"2024-05-09T08:56:00.687840+0000","last_became_peered":"2024-05-09T08:56:00.687840+0000","last_unstale":"2024-05-09T08:58:43.492637+0000","last_undegraded":"2024-05-09T08:58:43.492637+0000","last_fullsized":"2024-05-09T08:58:43.492637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:43.492526+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:43.492526+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.492526+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:29:23.103031+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":160,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:53.550990+0000","last_change":"2024-05-09T08:58:53.551055+0000","last_active":"2024-05-09T08:58:53.550990+0000","last_peered":"2024-05-09T08:58:53.550990+0000","last_clean":"2024-05-09T08:58:53.550990+0000","last_became_active":"2024-05-09T08:58:14.271699+0000","last_became_peered":"2024-05-09T08:58:14.271699+0000","last_unstale":"2024-05-09T08:58:53.550990+0000","last_undegraded":"2024-05-09T08:58:53.550990+0000","last_fullsized":"2024-05-09T08:58:53.550990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:53.550856+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:53.550856+0000","last_clean_scrub_stamp":"2024-05-09T08:58:53.550856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:08.280734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"83'25","reported_seq":93,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:40.067472+0000","last_change":"2024-05-09T08:58:40.067503+0000","last_active":"2024-05-09T08:58:40.067472+0000","last_peered":"2024-05-09T08:58:40.067472+0000","last_clean":"2024-05-09T08:58:40.067472+0000","last_became_active":"2024-05-09T08:58:13.285239+0000","last_became_peered":"2024-05-09T08:58:13.285239+0000","last_unstale":"2024-05-09T08:58:40.067472+0000","last_undegraded":"2024-05-09T08:58:40.067472+0000","last_fullsized":"2024-05-09T08:58:40.067472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'25","last_scrub_stamp":"2024-05-09T08:58:40.067400+0000","last_deep_scrub":"83'25","last_deep_scrub_stamp":"2024-05-09T08:58:40.067400+0000","last_clean_scrub_stamp":"2024-05-09T08:58:40.067400+0000","objects_scrubbed":8,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:04:02.122535+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.921058+0000","last_change":"2024-05-09T08:58:46.921107+0000","last_active":"2024-05-09T08:58:46.921058+0000","last_peered":"2024-05-09T08:58:46.921058+0000","last_clean":"2024-05-09T08:58:46.921058+0000","last_became_active":"2024-05-09T08:56:01.694786+0000","last_became_peered":"2024-05-09T08:56:01.694786+0000","last_unstale":"2024-05-09T08:58:46.921058+0000","last_undegraded":"2024-05-09T08:58:46.921058+0000","last_fullsized":"2024-05-09T08:58:46.921058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:46.920954+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:46.920954+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.920954+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:13:29.050500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"83'4","reported_seq":175,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.948432+0000","last_change":"2024-05-09T08:58:45.948465+0000","last_active":"2024-05-09T08:58:45.948432+0000","last_peered":"2024-05-09T08:58:45.948432+0000","last_clean":"2024-05-09T08:58:45.948432+0000","last_became_active":"2024-05-09T08:58:14.266799+0000","last_became_peered":"2024-05-09T08:58:14.266799+0000","last_unstale":"2024-05-09T08:58:45.948432+0000","last_undegraded":"2024-05-09T08:58:45.948432+0000","last_fullsized":"2024-05-09T08:58:45.948432+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:45.948320+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:45.948320+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.948320+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:56:10.018286+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:52.949362+0000","last_change":"2024-05-09T08:58:52.949391+0000","last_active":"2024-05-09T08:58:52.949362+0000","last_peered":"2024-05-09T08:58:52.949362+0000","last_clean":"2024-05-09T08:58:52.949362+0000","last_became_active":"2024-05-09T08:56:01.702692+0000","last_became_peered":"2024-05-09T08:56:01.702692+0000","last_unstale":"2024-05-09T08:58:52.949362+0000","last_undegraded":"2024-05-09T08:58:52.949362+0000","last_fullsized":"2024-05-09T08:58:52.949362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:52.949269+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:52.949269+0000","last_clean_scrub_stamp":"2024-05-09T08:58:52.949269+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:27:46.234318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'7","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:48.068660+0000","last_change":"2024-05-09T08:58:48.068716+0000","last_active":"2024-05-09T08:58:48.068660+0000","last_peered":"2024-05-09T08:58:48.068660+0000","last_clean":"2024-05-09T08:58:48.068660+0000","last_became_active":"2024-05-09T08:58:30.397753+0000","last_became_peered":"2024-05-09T08:58:30.397753+0000","last_unstale":"2024-05-09T08:58:48.068660+0000","last_undegraded":"2024-05-09T08:58:48.068660+0000","last_fullsized":"2024-05-09T08:58:48.068660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:48.068524+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:48.068524+0000","last_clean_scrub_stamp":"2024-05-09T08:58:48.068524+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:09.292358+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.964953+0000","last_change":"2024-05-09T08:58:47.964981+0000","last_active":"2024-05-09T08:58:47.964953+0000","last_peered":"2024-05-09T08:58:47.964953+0000","last_clean":"2024-05-09T08:58:47.964953+0000","last_became_active":"2024-05-09T08:56:01.695418+0000","last_became_peered":"2024-05-09T08:56:01.695418+0000","last_unstale":"2024-05-09T08:58:47.964953+0000","last_undegraded":"2024-05-09T08:58:47.964953+0000","last_fullsized":"2024-05-09T08:58:47.964953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:47.964850+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:47.964850+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.964850+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:00:00.835326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'4","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.146050+0000","last_change":"2024-05-09T08:58:43.146077+0000","last_active":"2024-05-09T08:58:43.146050+0000","last_peered":"2024-05-09T08:58:43.146050+0000","last_clean":"2024-05-09T08:58:43.146050+0000","last_became_active":"2024-05-09T08:56:00.695322+0000","last_became_peered":"2024-05-09T08:56:00.695322+0000","last_unstale":"2024-05-09T08:58:43.146050+0000","last_undegraded":"2024-05-09T08:58:43.146050+0000","last_fullsized":"2024-05-09T08:58:43.146050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'4","last_scrub_stamp":"2024-05-09T08:58:43.145955+0000","last_deep_scrub":"83'4","last_deep_scrub_stamp":"2024-05-09T08:58:43.145955+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.145955+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:37:46.865705+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'6","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.057212+0000","last_change":"2024-05-09T08:58:45.057255+0000","last_active":"2024-05-09T08:58:45.057212+0000","last_peered":"2024-05-09T08:58:45.057212+0000","last_clean":"2024-05-09T08:58:45.057212+0000","last_became_active":"2024-05-09T08:56:01.695435+0000","last_became_peered":"2024-05-09T08:56:01.695435+0000","last_unstale":"2024-05-09T08:58:45.057212+0000","last_undegraded":"2024-05-09T08:58:45.057212+0000","last_fullsized":"2024-05-09T08:58:45.057212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:45.057070+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:45.057070+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.057070+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:35:00.843712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"83'8","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.045569+0000","last_change":"2024-05-09T08:58:42.045595+0000","last_active":"2024-05-09T08:58:42.045569+0000","last_peered":"2024-05-09T08:58:42.045569+0000","last_clean":"2024-05-09T08:58:42.045569+0000","last_became_active":"2024-05-09T08:58:13.287686+0000","last_became_peered":"2024-05-09T08:58:13.287686+0000","last_unstale":"2024-05-09T08:58:42.045569+0000","last_undegraded":"2024-05-09T08:58:42.045569+0000","last_fullsized":"2024-05-09T08:58:42.045569+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:42.045496+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:42.045496+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.045496+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:01.586777+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:51.973711+0000","last_change":"2024-05-09T08:58:51.973737+0000","last_active":"2024-05-09T08:58:51.973711+0000","last_peered":"2024-05-09T08:58:51.973711+0000","last_clean":"2024-05-09T08:58:51.973711+0000","last_became_active":"2024-05-09T08:56:01.700357+0000","last_became_peered":"2024-05-09T08:56:01.700357+0000","last_unstale":"2024-05-09T08:58:51.973711+0000","last_undegraded":"2024-05-09T08:58:51.973711+0000","last_fullsized":"2024-05-09T08:58:51.973711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:51.973611+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:51.973611+0000","last_clean_scrub_stamp":"2024-05-09T08:58:51.973611+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:12.145432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'119","reported_seq":331,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:57.929659+0000","last_change":"2024-05-09T08:58:57.929703+0000","last_active":"2024-05-09T08:58:57.929659+0000","last_peered":"2024-05-09T08:58:57.929659+0000","last_clean":"2024-05-09T08:58:57.929659+0000","last_became_active":"2024-05-09T08:58:17.324130+0000","last_became_peered":"2024-05-09T08:58:17.324130+0000","last_unstale":"2024-05-09T08:58:57.929659+0000","last_undegraded":"2024-05-09T08:58:57.929659+0000","last_fullsized":"2024-05-09T08:58:57.929659+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'119","last_scrub_stamp":"2024-05-09T08:58:57.929564+0000","last_deep_scrub":"83'119","last_deep_scrub_stamp":"2024-05-09T08:58:57.929564+0000","last_clean_scrub_stamp":"2024-05-09T08:58:57.929564+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:16.521229+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"83'11","reported_seq":165,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.033243+0000","last_change":"2024-05-09T08:58:50.033284+0000","last_active":"2024-05-09T08:58:50.033243+0000","last_peered":"2024-05-09T08:58:50.033243+0000","last_clean":"2024-05-09T08:58:50.033243+0000","last_became_active":"2024-05-09T08:58:13.296720+0000","last_became_peered":"2024-05-09T08:58:13.296720+0000","last_unstale":"2024-05-09T08:58:50.033243+0000","last_undegraded":"2024-05-09T08:58:50.033243+0000","last_fullsized":"2024-05-09T08:58:50.033243+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:50.033155+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:50.033155+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.033155+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:12:15.247951+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.319906+0000","last_change":"2024-05-09T08:58:47.319933+0000","last_active":"2024-05-09T08:58:47.319906+0000","last_peered":"2024-05-09T08:58:47.319906+0000","last_clean":"2024-05-09T08:58:47.319906+0000","last_became_active":"2024-05-09T08:56:01.700520+0000","last_became_peered":"2024-05-09T08:56:01.700520+0000","last_unstale":"2024-05-09T08:58:47.319906+0000","last_undegraded":"2024-05-09T08:58:47.319906+0000","last_fullsized":"2024-05-09T08:58:47.319906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:47.319812+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:47.319812+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.319812+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:45.084238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"83'5","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.956828+0000","last_change":"2024-05-09T08:58:45.956858+0000","last_active":"2024-05-09T08:58:45.956828+0000","last_peered":"2024-05-09T08:58:45.956828+0000","last_clean":"2024-05-09T08:58:45.956828+0000","last_became_active":"2024-05-09T08:56:00.696975+0000","last_became_peered":"2024-05-09T08:56:00.696975+0000","last_unstale":"2024-05-09T08:58:45.956828+0000","last_undegraded":"2024-05-09T08:58:45.956828+0000","last_fullsized":"2024-05-09T08:58:45.956828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:45.956728+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:45.956728+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.956728+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:47:53.320576+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":56,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:39.101028+0000","last_change":"2024-05-09T08:58:39.101050+0000","last_active":"2024-05-09T08:58:39.101028+0000","last_peered":"2024-05-09T08:58:39.101028+0000","last_clean":"2024-05-09T08:58:39.101028+0000","last_became_active":"2024-05-09T08:58:14.260340+0000","last_became_peered":"2024-05-09T08:58:14.260340+0000","last_unstale":"2024-05-09T08:58:39.101028+0000","last_undegraded":"2024-05-09T08:58:39.101028+0000","last_fullsized":"2024-05-09T08:58:39.101028+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:39.100930+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:39.100930+0000","last_clean_scrub_stamp":"2024-05-09T08:58:39.100930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:03:22.794987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"83'11","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:46.074269+0000","last_change":"2024-05-09T08:58:46.074317+0000","last_active":"2024-05-09T08:58:46.074269+0000","last_peered":"2024-05-09T08:58:46.074269+0000","last_clean":"2024-05-09T08:58:46.074269+0000","last_became_active":"2024-05-09T08:58:13.293489+0000","last_became_peered":"2024-05-09T08:58:13.293489+0000","last_unstale":"2024-05-09T08:58:46.074269+0000","last_undegraded":"2024-05-09T08:58:46.074269+0000","last_fullsized":"2024-05-09T08:58:46.074269+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:46.074191+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:46.074191+0000","last_clean_scrub_stamp":"2024-05-09T08:58:46.074191+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:28:55.681524+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":161,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.146982+0000","last_change":"2024-05-09T08:58:50.147036+0000","last_active":"2024-05-09T08:58:50.146982+0000","last_peered":"2024-05-09T08:58:50.146982+0000","last_clean":"2024-05-09T08:58:50.146982+0000","last_became_active":"2024-05-09T08:58:13.287146+0000","last_became_peered":"2024-05-09T08:58:13.287146+0000","last_unstale":"2024-05-09T08:58:50.146982+0000","last_undegraded":"2024-05-09T08:58:50.146982+0000","last_fullsized":"2024-05-09T08:58:50.146982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:50.146852+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:50.146852+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.146852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:34:00.187376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":116,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:51.049769+0000","last_change":"2024-05-09T08:58:51.049834+0000","last_active":"2024-05-09T08:58:51.049769+0000","last_peered":"2024-05-09T08:58:51.049769+0000","last_clean":"2024-05-09T08:58:51.049769+0000","last_became_active":"2024-05-09T08:58:24.363124+0000","last_became_peered":"2024-05-09T08:58:24.363124+0000","last_unstale":"2024-05-09T08:58:51.049769+0000","last_undegraded":"2024-05-09T08:58:51.049769+0000","last_fullsized":"2024-05-09T08:58:51.049769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:51.049666+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:51.049666+0000","last_clean_scrub_stamp":"2024-05-09T08:58:51.049666+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:25:34.725604+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":164,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.966989+0000","last_change":"2024-05-09T08:58:44.967018+0000","last_active":"2024-05-09T08:58:44.966989+0000","last_peered":"2024-05-09T08:58:44.966989+0000","last_clean":"2024-05-09T08:58:44.966989+0000","last_became_active":"2024-05-09T08:58:14.266764+0000","last_became_peered":"2024-05-09T08:58:14.266764+0000","last_unstale":"2024-05-09T08:58:44.966989+0000","last_undegraded":"2024-05-09T08:58:44.966989+0000","last_fullsized":"2024-05-09T08:58:44.966989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:44.966883+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:44.966883+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.966883+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:39:15.050738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:52.141538+0000","last_change":"2024-05-09T08:58:52.141570+0000","last_active":"2024-05-09T08:58:52.141538+0000","last_peered":"2024-05-09T08:58:52.141538+0000","last_clean":"2024-05-09T08:58:52.141538+0000","last_became_active":"2024-05-09T08:58:14.270593+0000","last_became_peered":"2024-05-09T08:58:14.270593+0000","last_unstale":"2024-05-09T08:58:52.141538+0000","last_undegraded":"2024-05-09T08:58:52.141538+0000","last_fullsized":"2024-05-09T08:58:52.141538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'1","last_scrub_stamp":"2024-05-09T08:58:52.141426+0000","last_deep_scrub":"20'1","last_deep_scrub_stamp":"2024-05-09T08:58:52.141426+0000","last_clean_scrub_stamp":"2024-05-09T08:58:52.141426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:53.129389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.050070+0000","last_change":"2024-05-09T08:58:44.050104+0000","last_active":"2024-05-09T08:58:44.050070+0000","last_peered":"2024-05-09T08:58:44.050070+0000","last_clean":"2024-05-09T08:58:44.050070+0000","last_became_active":"2024-05-09T08:56:01.694059+0000","last_became_peered":"2024-05-09T08:56:01.694059+0000","last_unstale":"2024-05-09T08:58:44.050070+0000","last_undegraded":"2024-05-09T08:58:44.050070+0000","last_fullsized":"2024-05-09T08:58:44.050070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:44.049968+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:44.049968+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.049968+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:17:40.753571+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"83'7","reported_seq":180,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:55.954031+0000","last_change":"2024-05-09T08:58:55.954065+0000","last_active":"2024-05-09T08:58:55.954031+0000","last_peered":"2024-05-09T08:58:55.954031+0000","last_clean":"2024-05-09T08:58:55.954031+0000","last_became_active":"2024-05-09T08:58:14.269544+0000","last_became_peered":"2024-05-09T08:58:14.269544+0000","last_unstale":"2024-05-09T08:58:55.954031+0000","last_undegraded":"2024-05-09T08:58:55.954031+0000","last_fullsized":"2024-05-09T08:58:55.954031+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:55.953946+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:55.953946+0000","last_clean_scrub_stamp":"2024-05-09T08:58:55.953946+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:27:27.098257+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":162,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:49.138917+0000","last_change":"2024-05-09T08:58:49.138947+0000","last_active":"2024-05-09T08:58:49.138917+0000","last_peered":"2024-05-09T08:58:49.138917+0000","last_clean":"2024-05-09T08:58:49.138917+0000","last_became_active":"2024-05-09T08:58:13.285858+0000","last_became_peered":"2024-05-09T08:58:13.285858+0000","last_unstale":"2024-05-09T08:58:49.138917+0000","last_undegraded":"2024-05-09T08:58:49.138917+0000","last_fullsized":"2024-05-09T08:58:49.138917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:49.138823+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:49.138823+0000","last_clean_scrub_stamp":"2024-05-09T08:58:49.138823+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:41.039433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"83'20","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:40.124928+0000","last_change":"2024-05-09T08:58:40.124989+0000","last_active":"2024-05-09T08:58:40.124928+0000","last_peered":"2024-05-09T08:58:40.124928+0000","last_clean":"2024-05-09T08:58:40.124928+0000","last_became_active":"2024-05-09T08:56:00.690401+0000","last_became_peered":"2024-05-09T08:56:00.690401+0000","last_unstale":"2024-05-09T08:58:40.124928+0000","last_undegraded":"2024-05-09T08:58:40.124928+0000","last_fullsized":"2024-05-09T08:58:40.124928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'20","last_scrub_stamp":"2024-05-09T08:58:40.124837+0000","last_deep_scrub":"83'20","last_deep_scrub_stamp":"2024-05-09T08:58:40.124837+0000","last_clean_scrub_stamp":"2024-05-09T08:58:40.124837+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:21:44.853994+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.259404+0000","last_change":"2024-05-09T08:58:45.259484+0000","last_active":"2024-05-09T08:58:45.259404+0000","last_peered":"2024-05-09T08:58:45.259404+0000","last_clean":"2024-05-09T08:58:45.259404+0000","last_became_active":"2024-05-09T08:56:01.697298+0000","last_became_peered":"2024-05-09T08:56:01.697298+0000","last_unstale":"2024-05-09T08:58:45.259404+0000","last_undegraded":"2024-05-09T08:58:45.259404+0000","last_fullsized":"2024-05-09T08:58:45.259404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:45.259296+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:45.259296+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.259296+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:56:56.714739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'12","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.513707+0000","last_change":"2024-05-09T08:58:45.513751+0000","last_active":"2024-05-09T08:58:45.513707+0000","last_peered":"2024-05-09T08:58:45.513707+0000","last_clean":"2024-05-09T08:58:45.513707+0000","last_became_active":"2024-05-09T08:56:00.693585+0000","last_became_peered":"2024-05-09T08:56:00.693585+0000","last_unstale":"2024-05-09T08:58:45.513707+0000","last_undegraded":"2024-05-09T08:58:45.513707+0000","last_fullsized":"2024-05-09T08:58:45.513707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'12","last_scrub_stamp":"2024-05-09T08:58:45.513614+0000","last_deep_scrub":"83'12","last_deep_scrub_stamp":"2024-05-09T08:58:45.513614+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.513614+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:21.719132+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":58,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.131532+0000","last_change":"2024-05-09T08:58:44.131606+0000","last_active":"2024-05-09T08:58:44.131532+0000","last_peered":"2024-05-09T08:58:44.131532+0000","last_clean":"2024-05-09T08:58:44.131532+0000","last_became_active":"2024-05-09T08:58:14.266031+0000","last_became_peered":"2024-05-09T08:58:14.266031+0000","last_unstale":"2024-05-09T08:58:44.131532+0000","last_undegraded":"2024-05-09T08:58:44.131532+0000","last_fullsized":"2024-05-09T08:58:44.131532+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:44.131428+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:44.131428+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.131428+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:09.852974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'3","reported_seq":168,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:56.970505+0000","last_change":"2024-05-09T08:58:56.970539+0000","last_active":"2024-05-09T08:58:56.970505+0000","last_peered":"2024-05-09T08:58:56.970505+0000","last_clean":"2024-05-09T08:58:56.970505+0000","last_became_active":"2024-05-09T08:58:14.269581+0000","last_became_peered":"2024-05-09T08:58:14.269581+0000","last_unstale":"2024-05-09T08:58:56.970505+0000","last_undegraded":"2024-05-09T08:58:56.970505+0000","last_fullsized":"2024-05-09T08:58:56.970505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'3","last_scrub_stamp":"2024-05-09T08:58:56.970360+0000","last_deep_scrub":"83'3","last_deep_scrub_stamp":"2024-05-09T08:58:56.970360+0000","last_clean_scrub_stamp":"2024-05-09T08:58:56.970360+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:53:58.497801+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":156,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.938796+0000","last_change":"2024-05-09T08:58:43.938823+0000","last_active":"2024-05-09T08:58:43.938796+0000","last_peered":"2024-05-09T08:58:43.938796+0000","last_clean":"2024-05-09T08:58:43.938796+0000","last_became_active":"2024-05-09T08:58:13.296675+0000","last_became_peered":"2024-05-09T08:58:13.296675+0000","last_unstale":"2024-05-09T08:58:43.938796+0000","last_undegraded":"2024-05-09T08:58:43.938796+0000","last_fullsized":"2024-05-09T08:58:43.938796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'2","last_scrub_stamp":"2024-05-09T08:58:43.938712+0000","last_deep_scrub":"83'2","last_deep_scrub_stamp":"2024-05-09T08:58:43.938712+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.938712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:56:33.210881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"83'11","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.132610+0000","last_change":"2024-05-09T08:58:42.132662+0000","last_active":"2024-05-09T08:58:42.132610+0000","last_peered":"2024-05-09T08:58:42.132610+0000","last_clean":"2024-05-09T08:58:42.132610+0000","last_became_active":"2024-05-09T08:56:00.692688+0000","last_became_peered":"2024-05-09T08:56:00.692688+0000","last_unstale":"2024-05-09T08:58:42.132610+0000","last_undegraded":"2024-05-09T08:58:42.132610+0000","last_fullsized":"2024-05-09T08:58:42.132610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'11","last_scrub_stamp":"2024-05-09T08:58:42.132519+0000","last_deep_scrub":"83'11","last_deep_scrub_stamp":"2024-05-09T08:58:42.132519+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.132519+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:17:28.328808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:42.267952+0000","last_change":"2024-05-09T08:58:42.267983+0000","last_active":"2024-05-09T08:58:42.267952+0000","last_peered":"2024-05-09T08:58:42.267952+0000","last_clean":"2024-05-09T08:58:42.267952+0000","last_became_active":"2024-05-09T08:56:01.692767+0000","last_became_peered":"2024-05-09T08:56:01.692767+0000","last_unstale":"2024-05-09T08:58:42.267952+0000","last_undegraded":"2024-05-09T08:58:42.267952+0000","last_fullsized":"2024-05-09T08:58:42.267952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:42.267866+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:42.267866+0000","last_clean_scrub_stamp":"2024-05-09T08:58:42.267866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:08:13.101975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"83'6","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.117180+0000","last_change":"2024-05-09T08:58:44.117237+0000","last_active":"2024-05-09T08:58:44.117180+0000","last_peered":"2024-05-09T08:58:44.117180+0000","last_clean":"2024-05-09T08:58:44.117180+0000","last_became_active":"2024-05-09T08:56:00.696358+0000","last_became_peered":"2024-05-09T08:56:00.696358+0000","last_unstale":"2024-05-09T08:58:44.117180+0000","last_undegraded":"2024-05-09T08:58:44.117180+0000","last_fullsized":"2024-05-09T08:58:44.117180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:44.117039+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:44.117039+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.117039+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:48.435290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":163,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:50.330448+0000","last_change":"2024-05-09T08:58:50.330475+0000","last_active":"2024-05-09T08:58:50.330448+0000","last_peered":"2024-05-09T08:58:50.330448+0000","last_clean":"2024-05-09T08:58:50.330448+0000","last_became_active":"2024-05-09T08:58:14.268224+0000","last_became_peered":"2024-05-09T08:58:14.268224+0000","last_unstale":"2024-05-09T08:58:50.330448+0000","last_undegraded":"2024-05-09T08:58:50.330448+0000","last_fullsized":"2024-05-09T08:58:50.330448+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:50.330359+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:50.330359+0000","last_clean_scrub_stamp":"2024-05-09T08:58:50.330359+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:19:55.641063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'7","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:43.068834+0000","last_change":"2024-05-09T08:58:43.068873+0000","last_active":"2024-05-09T08:58:43.068834+0000","last_peered":"2024-05-09T08:58:43.068834+0000","last_clean":"2024-05-09T08:58:43.068834+0000","last_became_active":"2024-05-09T08:56:00.693645+0000","last_became_peered":"2024-05-09T08:56:00.693645+0000","last_unstale":"2024-05-09T08:58:43.068834+0000","last_undegraded":"2024-05-09T08:58:43.068834+0000","last_fullsized":"2024-05-09T08:58:43.068834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'7","last_scrub_stamp":"2024-05-09T08:58:43.068731+0000","last_deep_scrub":"83'7","last_deep_scrub_stamp":"2024-05-09T08:58:43.068731+0000","last_clean_scrub_stamp":"2024-05-09T08:58:43.068731+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:37:01.771142+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.507766+0000","last_change":"2024-05-09T08:58:47.507800+0000","last_active":"2024-05-09T08:58:47.507766+0000","last_peered":"2024-05-09T08:58:47.507766+0000","last_clean":"2024-05-09T08:58:47.507766+0000","last_became_active":"2024-05-09T08:56:01.686904+0000","last_became_peered":"2024-05-09T08:56:01.686904+0000","last_unstale":"2024-05-09T08:58:47.507766+0000","last_undegraded":"2024-05-09T08:58:47.507766+0000","last_fullsized":"2024-05-09T08:58:47.507766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:47.507659+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:47.507659+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.507659+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:01:53.707447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.076523+0000","last_change":"2024-05-09T08:58:45.076553+0000","last_active":"2024-05-09T08:58:45.076523+0000","last_peered":"2024-05-09T08:58:45.076523+0000","last_clean":"2024-05-09T08:58:45.076523+0000","last_became_active":"2024-05-09T08:58:13.294031+0000","last_became_peered":"2024-05-09T08:58:13.294031+0000","last_unstale":"2024-05-09T08:58:45.076523+0000","last_undegraded":"2024-05-09T08:58:45.076523+0000","last_fullsized":"2024-05-09T08:58:45.076523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'15","last_scrub_stamp":"2024-05-09T08:58:45.076427+0000","last_deep_scrub":"83'15","last_deep_scrub_stamp":"2024-05-09T08:58:45.076427+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.076427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:34:31.156985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"83'8","reported_seq":155,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:45.007326+0000","last_change":"2024-05-09T08:58:45.007364+0000","last_active":"2024-05-09T08:58:45.007326+0000","last_peered":"2024-05-09T08:58:45.007326+0000","last_clean":"2024-05-09T08:58:45.007326+0000","last_became_active":"2024-05-09T08:56:00.696732+0000","last_became_peered":"2024-05-09T08:56:00.696732+0000","last_unstale":"2024-05-09T08:58:45.007326+0000","last_undegraded":"2024-05-09T08:58:45.007326+0000","last_fullsized":"2024-05-09T08:58:45.007326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'8","last_scrub_stamp":"2024-05-09T08:58:45.007235+0000","last_deep_scrub":"83'8","last_deep_scrub_stamp":"2024-05-09T08:58:45.007235+0000","last_clean_scrub_stamp":"2024-05-09T08:58:45.007235+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:30:04.769314+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":116,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:55.066004+0000","last_change":"2024-05-09T08:58:55.066032+0000","last_active":"2024-05-09T08:58:55.066004+0000","last_peered":"2024-05-09T08:58:55.066004+0000","last_clean":"2024-05-09T08:58:55.066004+0000","last_became_active":"2024-05-09T08:58:24.726036+0000","last_became_peered":"2024-05-09T08:58:24.726036+0000","last_unstale":"2024-05-09T08:58:55.066004+0000","last_undegraded":"2024-05-09T08:58:55.066004+0000","last_fullsized":"2024-05-09T08:58:55.066004+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'9","last_scrub_stamp":"2024-05-09T08:58:55.065911+0000","last_deep_scrub":"83'9","last_deep_scrub_stamp":"2024-05-09T08:58:55.065911+0000","last_clean_scrub_stamp":"2024-05-09T08:58:55.065911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:51:47.898840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"83'20","reported_seq":167,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:44.048289+0000","last_change":"2024-05-09T08:58:44.048336+0000","last_active":"2024-05-09T08:58:44.048289+0000","last_peered":"2024-05-09T08:58:44.048289+0000","last_clean":"2024-05-09T08:58:44.048289+0000","last_became_active":"2024-05-09T08:56:00.696597+0000","last_became_peered":"2024-05-09T08:56:00.696597+0000","last_unstale":"2024-05-09T08:58:44.048289+0000","last_undegraded":"2024-05-09T08:58:44.048289+0000","last_fullsized":"2024-05-09T08:58:44.048289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'20","last_scrub_stamp":"2024-05-09T08:58:44.048203+0000","last_deep_scrub":"83'20","last_deep_scrub_stamp":"2024-05-09T08:58:44.048203+0000","last_clean_scrub_stamp":"2024-05-09T08:58:44.048203+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:46.496123+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":148,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:47.127781+0000","last_change":"2024-05-09T08:58:47.127814+0000","last_active":"2024-05-09T08:58:47.127781+0000","last_peered":"2024-05-09T08:58:47.127781+0000","last_clean":"2024-05-09T08:58:47.127781+0000","last_became_active":"2024-05-09T08:58:14.268398+0000","last_became_peered":"2024-05-09T08:58:14.268398+0000","last_unstale":"2024-05-09T08:58:47.127781+0000","last_undegraded":"2024-05-09T08:58:47.127781+0000","last_fullsized":"2024-05-09T08:58:47.127781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'6","last_scrub_stamp":"2024-05-09T08:58:47.127687+0000","last_deep_scrub":"83'6","last_deep_scrub_stamp":"2024-05-09T08:58:47.127687+0000","last_clean_scrub_stamp":"2024-05-09T08:58:47.127687+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:17:44.122067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'2","reported_seq":159,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:52.342360+0000","last_change":"2024-05-09T08:58:52.342385+0000","last_active":"2024-05-09T08:58:52.342360+0000","last_peered":"2024-05-09T08:58:52.342360+0000","last_clean":"2024-05-09T08:58:52.342360+0000","last_became_active":"2024-05-09T08:58:14.271721+0000","last_became_peered":"2024-05-09T08:58:14.271721+0000","last_unstale":"2024-05-09T08:58:52.342360+0000","last_undegraded":"2024-05-09T08:58:52.342360+0000","last_fullsized":"2024-05-09T08:58:52.342360+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'2","last_scrub_stamp":"2024-05-09T08:58:52.342259+0000","last_deep_scrub":"20'2","last_deep_scrub_stamp":"2024-05-09T08:58:52.342259+0000","last_clean_scrub_stamp":"2024-05-09T08:58:52.342259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:30:46.433455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":150,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:49.287894+0000","last_change":"2024-05-09T08:58:49.287955+0000","last_active":"2024-05-09T08:58:49.287894+0000","last_peered":"2024-05-09T08:58:49.287894+0000","last_clean":"2024-05-09T08:58:49.287894+0000","last_became_active":"2024-05-09T08:56:01.701285+0000","last_became_peered":"2024-05-09T08:56:01.701285+0000","last_unstale":"2024-05-09T08:58:49.287894+0000","last_undegraded":"2024-05-09T08:58:49.287894+0000","last_fullsized":"2024-05-09T08:58:49.287894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'5","last_scrub_stamp":"2024-05-09T08:58:49.287781+0000","last_deep_scrub":"83'5","last_deep_scrub_stamp":"2024-05-09T08:58:49.287781+0000","last_clean_scrub_stamp":"2024-05-09T08:58:49.287781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:42:51.196553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'19","reported_seq":71,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-09T08:58:49.103354+0000","last_change":"2024-05-09T08:58:49.103403+0000","last_active":"2024-05-09T08:58:49.103354+0000","last_peered":"2024-05-09T08:58:49.103354+0000","last_clean":"2024-05-09T08:58:49.103354+0000","last_became_active":"2024-05-09T08:58:14.266269+0000","last_became_peered":"2024-05-09T08:58:14.266269+0000","last_unstale":"2024-05-09T08:58:49.103354+0000","last_undegraded":"2024-05-09T08:58:49.103354+0000","last_fullsized":"2024-05-09T08:58:49.103354+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'19","last_scrub_stamp":"2024-05-09T08:58:49.103268+0000","last_deep_scrub":"83'19","last_deep_scrub_stamp":"2024-05-09T08:58:49.103268+0000","last_clean_scrub_stamp":"2024-05-09T08:58:49.103268+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:26.662779+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44671,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1311,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":89342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1609728,"data_stored":1050735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1044480,"data_stored":496209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1028096,"data_stored":461802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":1003520,"data_stored":459200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1052672,"data_stored":496943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":65,"seq":279172874252,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29800,"kb_used_data":992,"kb_used_omap":14,"kb_used_meta":28785,"kb_avail":94342040,"statfs":{"total":96636764160,"available":96606248960,"internally_reserved":0,"allocated":1015808,"data_stored":459635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14372,"internal_metadata":29476828},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1007616,"data_stored":459402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1601536,"data_stored":1049112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12777,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":39073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":38339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T08:59:02.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T08:59:02.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.852+0000 7effa2009640 1 Processor -- start 2024-05-09T08:59:02.851 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.852+0000 7effa2009640 1 -- start start 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7effa2009640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff9c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7effa2009640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff9c0c6fd0 con 0x7eff9c0c6b80 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7effa2009640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff9c072600 con 0x7eff9c154b90 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7effa2009640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff9c0728e0 con 0x7eff9c154f70 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9bfff640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7eff9c0c6b80 0x7eff9c071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41102/0 (socket says 172.21.15.49:41102) 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9bfff640 1 -- 172.21.15.49:0/2946987370 learned_addr learned my addr 172.21.15.49:0/2946987370 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9affd640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff9c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040663097 0 0) 0x7eff9c0c6fd0 con 0x7eff9c0c6b80 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff84003520 con 0x7eff9c0c6b80 2024-05-09T08:59:02.855 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1211039125 0 0) 0x7eff84003520 con 0x7eff9c0c6b80 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 msgr2=0x7eff9c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff9c0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 >> v1:172.21.15.183:6789/0 conn(0x7eff9c154b90 legacy=0x7eff9c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c0c5880 con 0x7eff9c0c6b80 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7eff940019f0 con 0x7eff9c0c6b80 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 >> v1:172.21.15.49:6789/0 conn(0x7eff9c0c6b80 legacy=0x7eff9c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.853+0000 7eff9a7fc640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff84004d40 0x7eff840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c0728e0 con 0x7eff84003f20 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff8400ba20 con 0x7eff84004d40 2024-05-09T08:59:02.856 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff8400bc20 con 0x7eff9c154f70 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9b7fe640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9bfff640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff84004d40 0x7eff840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 -- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 msgr2=0x7eff840039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c154f70 0x7eff840039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 -- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff84004d40 msgr2=0x7eff840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff84004d40 0x7eff840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.854+0000 7eff9affd640 1 -- 172.21.15.49:0/2946987370 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84003520 con 0x7eff84003f20 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7eff9affd640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff84004800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7eff90009ac0 tx=0x7eff84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60a3f7e3d768bed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.857 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90036070 con 0x7eff84003f20 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90031400 con 0x7eff84003f20 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7eff9a7fc640 1 -- 172.21.15.49:0/2946987370 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90009690 con 0x7eff84003f20 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7effa2009640 1 -- 172.21.15.49:0/2946987370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 msgr2=0x7eff84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7effa2009640 1 --2- 172.21.15.49:0/2946987370 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff84004800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7eff90009ac0 tx=0x7eff84005830 comp rx=0 tx=0).stop 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7eff9bfff640 1 -- 172.21.15.49:0/2946987370 reap_dead start 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7effa2009640 1 -- 172.21.15.49:0/2946987370 shutdown_connections 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.855+0000 7effa2009640 1 -- 172.21.15.49:0/2946987370 >> 172.21.15.49:0/2946987370 conn(0x7eff9c074840 msgr2=0x7eff9c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.856+0000 7effa2009640 1 -- 172.21.15.49:0/2946987370 shutdown_connections 2024-05-09T08:59:02.858 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.856+0000 7effa2009640 1 -- 172.21.15.49:0/2946987370 wait complete. 2024-05-09T08:59:02.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.857+0000 7effa2009640 1 Processor -- start 2024-05-09T08:59:02.861 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.857+0000 7effa2009640 1 -- start start 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 0x7eff9c147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c0ccf20 0x7eff9c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c0c6120 con 0x7eff84003f20 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c0c5e40 con 0x7eff9c1535f0 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7eff9c15af40 con 0x7eff9c0ccf20 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35024/0 (socket says 172.21.15.49:35024) 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 -- 172.21.15.49:0/4084267058 learned_addr learned my addr 172.21.15.49:0/4084267058 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9affd640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 0x7eff9c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9bfff640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c0ccf20 0x7eff9c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.862 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 -- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c0ccf20 msgr2=0x7eff9c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c0ccf20 0x7eff9c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 -- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 msgr2=0x7eff9c147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 0x7eff9c147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 -- 172.21.15.49:0/4084267058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c1f4150 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9affd640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 0x7eff9c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff9b7fe640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00f820 tx=0x7eff8c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0052e0 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c018070 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9c1f43d0 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00d970 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.858+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9c1f1a80 con 0x7eff84003f20 2024-05-09T08:59:02.863 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.859+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff9c04f660 con 0x7eff84003f20 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.860+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff8c00db10 con 0x7eff84003f20 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.860+0000 7eff83fff640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 0x7eff8806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.861+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7eff8c08ae50 con 0x7eff84003f20 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.861+0000 7eff9affd640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 0x7eff8806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.861+0000 7eff9affd640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 0x7eff8806e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff90033000 tx=0x7eff90008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:02.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:02.865+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7eff9c1f4150 con 0x7eff84003f20 2024-05-09T08:59:03.053 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.054+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7eff9c0cbde0 con 0x7eff84003f20 2024-05-09T08:59:03.056 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.057+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7eff8c01a310 con 0x7eff84003f20 2024-05-09T08:59:03.058 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.059+0000 7eff83fff640 1 -- 172.21.15.49:0/4084267058 <== mon.0 v2:172.21.15.49:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7eff8c005480 con 0x7eff84003f20 2024-05-09T08:59:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.060+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 msgr2=0x7eff8806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.060+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 0x7eff8806e320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7eff90033000 tx=0x7eff90008030 comp rx=0 tx=0).stop 2024-05-09T08:59:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.060+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 msgr2=0x7eff9c152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.060 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.060+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00f820 tx=0x7eff8c00fce0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 shutdown_connections 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7eff8806be60 0x7eff8806e320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff9c0ccf20 0x7eff9c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff9c1535f0 0x7eff9c147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 --2- 172.21.15.49:0/4084267058 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7eff84003f20 0x7eff9c152e20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 >> 172.21.15.49:0/4084267058 conn(0x7eff9c074840 msgr2=0x7eff9c158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 shutdown_connections 2024-05-09T08:59:03.061 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.061+0000 7effa2009640 1 -- 172.21.15.49:0/4084267058 wait complete. 2024-05-09T08:59:03.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T08:59:03.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:59:03.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.178+0000 7fadf6128640 1 Processor -- start 2024-05-09T08:59:03.178 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.179+0000 7fadf6128640 1 -- start start 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadf6128640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fadf00c6e00 0x7fadf00726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadf6128640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00c4110 con 0x7fadf0072300 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadf6128640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00c3b50 con 0x7fadf00c66c0 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadf6128640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7fadf00c3900 con 0x7fadf00c6e00 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadeffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fadf00c6e00 0x7fadf00726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadeffff640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7fadf00c6e00 0x7fadf00726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35028/0 (socket says 172.21.15.49:35028) 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadeffff640 1 -- 172.21.15.49:0/2167722543 learned_addr learned my addr 172.21.15.49:0/2167722543 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadef7fe640 1 --1- 172.21.15.49:0/2167722543 >> v1:172.21.15.49:6789/0 conn(0x7fadf0072300 0x7fadf01593b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41106/0 (socket says 172.21.15.49:41106) 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.180+0000 7fadeffff640 1 -- 172.21.15.49:0/2167722543 >> v1:172.21.15.183:6789/0 conn(0x7fadf00c66c0 legacy=0x7fadf006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:03.180 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadeffff640 1 -- 172.21.15.49:0/2167722543 >> v1:172.21.15.49:6789/0 conn(0x7fadf0072300 legacy=0x7fadf01593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadee7fc640 1 -- 172.21.15.49:0/2167722543 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 419674245 0 0) 0x7fadf00c4110 con 0x7fadf0072300 2024-05-09T08:59:03.181 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadeffff640 1 -- 172.21.15.49:0/2167722543 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf00c3e30 con 0x7fadf00c6e00 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadeffff640 1 --2- 172.21.15.49:0/2167722543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c6e00 0x7fadf00726e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587e3d022db74b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadee7fc640 1 -- 172.21.15.49:0/2167722543 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8036070 con 0x7fadf00c6e00 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadee7fc640 1 -- 172.21.15.49:0/2167722543 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fade8031600 con 0x7fadf00c6e00 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.181+0000 7fadee7fc640 1 -- 172.21.15.49:0/2167722543 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8031a10 con 0x7fadf00c6e00 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 -- 172.21.15.49:0/2167722543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c6e00 msgr2=0x7fadf00726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 --2- 172.21.15.49:0/2167722543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c6e00 0x7fadf00726e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade802a7c0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.182 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 -- 172.21.15.49:0/2167722543 shutdown_connections 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 --2- 172.21.15.49:0/2167722543 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c6e00 0x7fadf00726e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 -- 172.21.15.49:0/2167722543 >> 172.21.15.49:0/2167722543 conn(0x7fadf0074840 msgr2=0x7fadf0074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 -- 172.21.15.49:0/2167722543 shutdown_connections 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.182+0000 7fadf6128640 1 -- 172.21.15.49:0/2167722543 wait complete. 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.184+0000 7fadf6128640 1 Processor -- start 2024-05-09T08:59:03.183 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.184+0000 7fadf6128640 1 -- start start 2024-05-09T08:59:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 0x7fadf01f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf00c6e00 0x7fadf01402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fadf014f9e0 con 0x7fadf00c66c0 2024-05-09T08:59:03.186 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0158f20 con 0x7fadf00c6e00 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadf6128640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fadf014fb60 con 0x7fadf0072300 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.185+0000 7fadeeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35044/0 (socket says 172.21.15.49:35044) 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 -- 172.21.15.49:0/1240030336 learned_addr learned my addr 172.21.15.49:0/1240030336 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeffff640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf00c6e00 0x7fadf01402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadef7fe640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 0x7fadf01f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 -- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 msgr2=0x7fadf01f2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 0x7fadf01f2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 -- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf00c6e00 msgr2=0x7fadf01402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf00c6e00 0x7fadf01402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 -- 172.21.15.49:0/1240030336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf01e4760 con 0x7fadf00c66c0 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadef7fe640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 0x7fadf01f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:59:03.187 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadeeffd640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fade400c400 tx=0x7fade400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4017070 con 0x7fadf00c66c0 2024-05-09T08:59:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fade40052d0 con 0x7fadf00c66c0 2024-05-09T08:59:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf01e4a40 con 0x7fadf00c66c0 2024-05-09T08:59:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4003850 con 0x7fadf00c66c0 2024-05-09T08:59:03.188 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.186+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf01f0720 con 0x7fadf00c66c0 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.188+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf0068560 con 0x7fadf00c66c0 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.188+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fade401a070 con 0x7fadf00c66c0 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.188+0000 7fadd7fff640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 0x7faddc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.189+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fade408a220 con 0x7fadf00c66c0 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.189+0000 7fadef7fe640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 0x7faddc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.189 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.189+0000 7fadef7fe640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 0x7faddc06e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fade0000f60 tx=0x7fade0007c50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.192 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.193+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fade40584d0 con 0x7fadf00c66c0 2024-05-09T08:59:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.486+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadf00c4340 con 0x7fadf00c66c0 2024-05-09T08:59:03.485 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.486+0000 7fadd7fff640 1 -- 172.21.15.49:0/1240030336 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6912 (secure 0 0 0) 0x7fade405c7d0 con 0x7fadf00c66c0 2024-05-09T08:59:03.486 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:59:03.486 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:30.763288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4235,4238,4251,4241,4285]},"id":1}]} 2024-05-09T08:59:03.486 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 16 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.488+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 msgr2=0x7faddc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.488+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 0x7faddc06e320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fade0000f60 tx=0x7fade0007c50 comp rx=0 tx=0).stop 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.488+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 msgr2=0x7fadf0149eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.488+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fade400c400 tx=0x7fade400c8c0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 shutdown_connections 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7faddc06be60 0x7faddc06e320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf00c6e00 0x7fadf01402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.488 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fadf00c66c0 0x7fadf0149eb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 --2- 172.21.15.49:0/1240030336 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fadf0072300 0x7fadf01f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 >> 172.21.15.49:0/1240030336 conn(0x7fadf0074840 msgr2=0x7fadf014e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 shutdown_connections 2024-05-09T08:59:03.489 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.489+0000 7fadf6128640 1 -- 172.21.15.49:0/1240030336 wait complete. 2024-05-09T08:59:03.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T08:59:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.605+0000 7f38f0b61640 1 Processor -- start 2024-05-09T08:59:03.604 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.605+0000 7f38f0b61640 1 -- start start 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38f0b61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38f0b61640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38f0b61640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38ec15e260 con 0x7f38ec0cd520 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38f0b61640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f38ec151bd0 con 0x7f38ec075200 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38f0b61640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec15e520 con 0x7f38ec1517f0 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38e9d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38ea59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38ea59c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:54158/0 (socket says 172.21.15.49:54158) 2024-05-09T08:59:03.606 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38ead9d640 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f38ec0cd520 0x7f38ec0cc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.49:60116/0 (socket says 172.21.15.49:60116) 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38e9d9b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.49:46804/0 (socket says 172.21.15.49:46804) 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.606+0000 7f38ea59c640 1 -- 172.21.15.49:0/457804712 learned_addr learned my addr 172.21.15.49:0/457804712 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38ea59c640 1 -- 172.21.15.49:0/457804712 >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 msgr2=0x7f38ec0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38ea59c640 1 --2- 172.21.15.49:0/457804712 >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38ea59c640 1 -- 172.21.15.49:0/457804712 >> v1:172.21.15.183:6789/0 conn(0x7f38ec0cd520 legacy=0x7f38ec0cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38ea59c640 1 -- 172.21.15.49:0/457804712 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec15e800 con 0x7f38ec1517f0 2024-05-09T08:59:03.607 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38e9d9b640 1 --2- 172.21.15.49:0/457804712 >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.607+0000 7f38ea59c640 1 --2- 172.21.15.49:0/457804712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098a0 tx=0x7f38dc02a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b80215b841de1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38e959a640 1 -- 172.21.15.49:0/457804712 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc037070 con 0x7f38ec1517f0 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38e959a640 1 -- 172.21.15.49:0/457804712 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f38dc032640 con 0x7f38ec1517f0 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38e959a640 1 -- 172.21.15.49:0/457804712 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc032a30 con 0x7f38ec1517f0 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 -- 172.21.15.49:0/457804712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 msgr2=0x7f38ec15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 --2- 172.21.15.49:0/457804712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098a0 tx=0x7f38dc02a320 comp rx=0 tx=0).stop 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 -- 172.21.15.49:0/457804712 shutdown_connections 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 --2- 172.21.15.49:0/457804712 >> v2:172.21.15.183:3300/0 conn(0x7f38ec075200 0x7f38ec0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 --2- 172.21.15.49:0/457804712 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec1517f0 0x7f38ec15b6c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.609 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 -- 172.21.15.49:0/457804712 >> 172.21.15.49:0/457804712 conn(0x7f38ec06bd00 msgr2=0x7f38ec074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 -- 172.21.15.49:0/457804712 shutdown_connections 2024-05-09T08:59:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.608+0000 7f38f0b61640 1 -- 172.21.15.49:0/457804712 wait complete. 2024-05-09T08:59:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 Processor -- start 2024-05-09T08:59:03.610 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 -- start start 2024-05-09T08:59:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec0cd520 0x7f38ec1e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 0x7f38ec1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.612 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec15f0a0 con 0x7f38ec075200 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec15edc0 con 0x7f38ec1517f0 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38f0b61640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec0cbc70 con 0x7f38ec0cd520 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.610+0000 7f38ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35058/0 (socket says 172.21.15.49:35058) 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 -- 172.21.15.49:0/2477651790 learned_addr learned my addr 172.21.15.49:0/2477651790 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38e9d9b640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec0cd520 0x7f38ec1e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ead9d640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 0x7f38ec1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 -- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec0cd520 msgr2=0x7f38ec1e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec0cd520 0x7f38ec1e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 -- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 msgr2=0x7f38ec1f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 0x7f38ec1f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 -- 172.21.15.49:0/2477651790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec1f4190 con 0x7f38ec075200 2024-05-09T08:59:03.613 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ead9d640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 0x7f38ec1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38ea59c640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02a360 tx=0x7f38dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc037040 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f38dc042070 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0309d0 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec1f4430 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.611+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1f49f0 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.613+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec068560 con 0x7f38ec075200 2024-05-09T08:59:03.614 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.613+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38dc030b70 con 0x7f38ec075200 2024-05-09T08:59:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.614+0000 7f38daffd640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 0x7f38cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.614+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f38dc0ab7c0 con 0x7f38ec075200 2024-05-09T08:59:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.614+0000 7f38e9d9b640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 0x7f38cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.615 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.614+0000 7f38e9d9b640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 0x7f38cc06e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009560 tx=0x7f38e0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.617 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.618+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f38dc07a0c0 con 0x7f38ec075200 2024-05-09T08:59:03.839 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.840+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ec0caa00 con 0x7f38ec075200 2024-05-09T08:59:03.840 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.841+0000 7f38daffd640 1 -- 172.21.15.49:0/2477651790 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6912 (secure 0 0 0) 0x7f38dc0799c0 con 0x7f38ec075200 2024-05-09T08:59:03.840 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:59:03.840 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2559170747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2559170747},{"type":"v1","addr":"172.21.15.183:6837","nonce":2559170747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T08:56:01.049338+0000","modified":"2024-05-09T08:56:30.763288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4285,"mds_1":4251,"mds_2":4241,"mds_3":4238,"mds_4":4235},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4235":{"gid":4235,"name":"b","rank":4,"incarnation":15,"state":"up:active","state_seq":10,"addr":"172.21.15.183:6838/2318987144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2318987144},{"type":"v1","addr":"172.21.15.183:6838","nonce":2318987144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":13,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6839/1999269737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6835","nonce":1999269737},{"type":"v1","addr":"172.21.15.183:6839","nonce":1999269737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4241":{"gid":4241,"name":"e","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.49:6837/2879019682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6836","nonce":2879019682},{"type":"v1","addr":"172.21.15.49:6837","nonce":2879019682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4251":{"gid":4251,"name":"c","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.49:6834/204294090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6833","nonce":204294090},{"type":"v1","addr":"172.21.15.49:6834","nonce":204294090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4285":{"gid":4285,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.49:6835/2800676488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6832","nonce":2800676488},{"type":"v1","addr":"172.21.15.49:6835","nonce":2800676488}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4235,4238,4251,4241,4285]},"id":1}]} 2024-05-09T08:59:03.841 INFO:teuthology.orchestra.run.smithi049.stderr:dumped fsmap epoch 16 2024-05-09T08:59:03.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 msgr2=0x7f38cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.842 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 0x7f38cc06e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009560 tx=0x7f38e0009190 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 msgr2=0x7f38ec072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f38dc02a360 tx=0x7f38dc0046c0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 shutdown_connections 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f38cc06be60 0x7f38cc06e320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38ec1517f0 0x7f38ec1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f38ec0cd520 0x7f38ec1e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 --2- 172.21.15.49:0/2477651790 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f38ec075200 0x7f38ec072030 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 >> 172.21.15.49:0/2477651790 conn(0x7f38ec06bd00 msgr2=0x7f38ec074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 shutdown_connections 2024-05-09T08:59:03.843 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.843+0000 7f38f0b61640 1 -- 172.21.15.49:0/2477651790 wait complete. 2024-05-09T08:59:03.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T08:59:03.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.958+0000 7f293002c640 1 Processor -- start 2024-05-09T08:59:03.958 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.959+0000 7f293002c640 1 -- start start 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f293002c640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2928151e00 0x7f29280c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f293002c640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29280c8350 con 0x7f29280c9c00 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f293002c640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2928158490 con 0x7f29280c8780 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f293002c640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f29281521e0 con 0x7f2928151e00 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292d5c7640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f29280c9c00 0x7f29280c8e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41120/0 (socket says 172.21.15.49:41120) 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292d5c7640 1 -- 172.21.15.49:0/951754564 learned_addr learned my addr 172.21.15.49:0/951754564 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193222199 0 0) 0x7f2918001110 con 0x7f29280c9c00 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292ddc8640 1 --2- 172.21.15.49:0/951754564 >> v2:172.21.15.183:3300/0 conn(0x7f2928151e00 0x7f29280c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.961 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2914003520 con 0x7f29280c9c00 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987280809 0 0) 0x7f2914003520 con 0x7f29280c9c00 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 >> v2:172.21.15.183:3300/0 conn(0x7f2928151e00 msgr2=0x7f29280c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 --2- 172.21.15.49:0/951754564 >> v2:172.21.15.183:3300/0 conn(0x7f2928151e00 0x7f29280c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 >> v1:172.21.15.183:6789/0 conn(0x7f29280c8780 legacy=0x7f29281622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2928158750 con 0x7f29280c9c00 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7f2918001840 con 0x7f29280c9c00 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 >> v1:172.21.15.49:6789/0 conn(0x7f29280c9c00 legacy=0x7f29280c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29140039b0 0x7f2914003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2914004800 0x7f2914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.960+0000 7f292cdc6640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 0x7f291400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f29281521e0 con 0x7f29140051c0 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f291400bf80 con 0x7f29140039b0 2024-05-09T08:59:03.962 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f291400c180 con 0x7f2914004800 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 0x7f291400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292ddc8640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29140039b0 0x7f2914003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292d5c7640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2914004800 0x7f2914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 -- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2914004800 msgr2=0x7f2914004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2914004800 0x7f2914004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 -- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29140039b0 msgr2=0x7f2914003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29140039b0 0x7f2914003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.961+0000 7f292e5c9640 1 -- 172.21.15.49:0/951754564 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2914003520 con 0x7f29140051c0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f292e5c9640 1 -- 172.21.15.49:0/951754564 reap_dead start 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f292e5c9640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 0x7f291400b9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2920001f60 tx=0x7f2914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330374ff83df47fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920001b30 con 0x7f29140051c0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2920003b90 con 0x7f29140051c0 2024-05-09T08:59:03.963 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f292cdc6640 1 -- 172.21.15.49:0/951754564 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920005110 con 0x7f29140051c0 2024-05-09T08:59:03.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f293002c640 1 -- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 msgr2=0x7f291400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.966 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f293002c640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 0x7f291400b9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2920001f60 tx=0x7f2914005830 comp rx=0 tx=0).stop 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f293002c640 1 -- 172.21.15.49:0/951754564 shutdown_connections 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f293002c640 1 --2- 172.21.15.49:0/951754564 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f29140051c0 0x7f291400b9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.962+0000 7f293002c640 1 -- 172.21.15.49:0/951754564 >> 172.21.15.49:0/951754564 conn(0x7f2928074840 msgr2=0x7f2928074c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.963+0000 7f293002c640 1 -- 172.21.15.49:0/951754564 shutdown_connections 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.963+0000 7f293002c640 1 -- 172.21.15.49:0/951754564 wait complete. 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 Processor -- start 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 -- start start 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 0x7f29280c2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2928152380 0x7f2928153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f2928071bb0 con 0x7f2928153e40 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29280718d0 con 0x7f2928152380 2024-05-09T08:59:03.967 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.964+0000 7f293002c640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f2928071e90 con 0x7f29140051c0 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292ddc8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 0x7f29280c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292d5c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2928152380 0x7f2928153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35082/0 (socket says 172.21.15.49:35082) 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292ddc8640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 0x7f29280c2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.49:54192/0 (socket says 172.21.15.49:54192) 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 -- 172.21.15.49:0/962462325 learned_addr learned my addr 172.21.15.49:0/962462325 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 -- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 msgr2=0x7f29280c2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 0x7f29280c2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 -- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2928152380 msgr2=0x7f2928153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2928152380 0x7f2928153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 -- 172.21.15.49:0/962462325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292815ae50 con 0x7f2928153e40 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f292e5c9640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2920002050 tx=0x7f29200015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920006d50 con 0x7f2928153e40 2024-05-09T08:59:03.968 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f29200057e0 con 0x7f2928153e40 2024-05-09T08:59:03.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292815b130 con 0x7f2928153e40 2024-05-09T08:59:03.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.965+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292003e670 con 0x7f2928153e40 2024-05-09T08:59:03.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.966+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292815b6f0 con 0x7f2928153e40 2024-05-09T08:59:03.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.966+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2928068560 con 0x7f2928153e40 2024-05-09T08:59:03.969 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.967+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2920030070 con 0x7f2928153e40 2024-05-09T08:59:03.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.968+0000 7f291e7fc640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 0x7f290c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:03.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.968+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7f29200aacc0 con 0x7f2928153e40 2024-05-09T08:59:03.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.968+0000 7f292ddc8640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 0x7f290c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:03.970 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.969+0000 7f292ddc8640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 0x7f290c06e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2924009b40 tx=0x7f2924009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:03.971 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:03.973+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2920045020 con 0x7f2928153e40 2024-05-09T08:59:04.160 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.160+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29280ca580 con 0x7f2928153e40 2024-05-09T08:59:04.161 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.162+0000 7f291e7fc640 1 -- 172.21.15.49:0/962462325 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16838 (secure 0 0 0) 0x7f2920079690 con 0x7f2928153e40 2024-05-09T08:59:04.161 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-05-09T08:59:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":83,"fsid":"aab8b2de-6010-439c-b8fd-8bc72445265c","created":"2024-05-09T08:55:44.445697+0000","modified":"2024-05-09T08:58:30.390193+0000","last_up_change":"2024-05-09T08:58:12.270880+0000","last_in_change":"2024-05-09T08:58:13.272774+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T08:55:56.616182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T08:55:59.400583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T08:56:00.024628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b59d26bf-6806-4e49-b117-9d83d3a24243","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6801","nonce":1154213557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6802","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6803","nonce":1154213557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6806","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6807","nonce":1154213557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":1154213557},{"type":"v1","addr":"172.21.15.49:6805","nonce":1154213557}]},"public_addr":"172.21.15.49:6801/1154213557","cluster_addr":"172.21.15.49:6803/1154213557","heartbeat_back_addr":"172.21.15.49:6807/1154213557","heartbeat_front_addr":"172.21.15.49:6805/1154213557","state":["exists","up"]},{"osd":1,"uuid":"d579c6a2-0370-4c21-96fb-6420583dbe74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6817","nonce":3346547076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6819","nonce":3346547076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6823","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6825","nonce":3346547076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":3346547076},{"type":"v1","addr":"172.21.15.49:6821","nonce":3346547076}]},"public_addr":"172.21.15.49:6817/3346547076","cluster_addr":"172.21.15.49:6819/3346547076","heartbeat_back_addr":"172.21.15.49:6825/3346547076","heartbeat_front_addr":"172.21.15.49:6821/3346547076","state":["exists","up"]},{"osd":2,"uuid":"bb2ad0e5-2ec1-4647-bc42-3d20911e038a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":38,"up_from":65,"up_thru":82,"down_at":39,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6809","nonce":282207482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6810","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6811","nonce":282207482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6814","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6815","nonce":282207482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":282207482},{"type":"v1","addr":"172.21.15.49:6813","nonce":282207482}]},"public_addr":"172.21.15.49:6809/282207482","cluster_addr":"172.21.15.49:6811/282207482","heartbeat_back_addr":"172.21.15.49:6815/282207482","heartbeat_front_addr":"172.21.15.49:6813/282207482","state":["exists","up"]},{"osd":3,"uuid":"546a2935-7144-4c29-b647-e77d89072d60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6824","nonce":1536339711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6827","nonce":1536339711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6831","nonce":1536339711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":1536339711},{"type":"v1","addr":"172.21.15.49:6829","nonce":1536339711}]},"public_addr":"172.21.15.49:6824/1536339711","cluster_addr":"172.21.15.49:6827/1536339711","heartbeat_back_addr":"172.21.15.49:6831/1536339711","heartbeat_front_addr":"172.21.15.49:6829/1536339711","state":["exists","up"]},{"osd":4,"uuid":"f4bb828b-6036-4886-aec7-be53c00ac603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6809","nonce":173400960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6811","nonce":173400960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6815","nonce":173400960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":173400960},{"type":"v1","addr":"172.21.15.183:6813","nonce":173400960}]},"public_addr":"172.21.15.183:6809/173400960","cluster_addr":"172.21.15.183:6811/173400960","heartbeat_back_addr":"172.21.15.183:6815/173400960","heartbeat_front_addr":"172.21.15.183:6813/173400960","state":["exists","up"]},{"osd":5,"uuid":"41625c33-9f16-4279-b2aa-a63ac0aa6320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6801","nonce":4279483611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6803","nonce":4279483611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6807","nonce":4279483611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4279483611},{"type":"v1","addr":"172.21.15.183:6805","nonce":4279483611}]},"public_addr":"172.21.15.183:6801/4279483611","cluster_addr":"172.21.15.183:6803/4279483611","heartbeat_back_addr":"172.21.15.183:6807/4279483611","heartbeat_front_addr":"172.21.15.183:6805/4279483611","state":["exists","up"]},{"osd":6,"uuid":"9177552d-bc69-409e-9654-e030c3a1afba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6825","nonce":3521264583}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6827","nonce":3521264583}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6831","nonce":3521264583}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":3521264583},{"type":"v1","addr":"172.21.15.183:6829","nonce":3521264583}]},"public_addr":"172.21.15.183:6825/3521264583","cluster_addr":"172.21.15.183:6827/3521264583","heartbeat_back_addr":"172.21.15.183:6831/3521264583","heartbeat_front_addr":"172.21.15.183:6829/3521264583","state":["exists","up"]},{"osd":7,"uuid":"d3b1821c-1ed4-4286-b7de-030fda235fbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6817","nonce":2743896476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6819","nonce":2743896476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6823","nonce":2743896476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":2743896476},{"type":"v1","addr":"172.21.15.183:6821","nonce":2743896476}]},"public_addr":"172.21.15.183:6817/2743896476","cluster_addr":"172.21.15.183:6819/2743896476","heartbeat_back_addr":"172.21.15.183:6823/2743896476","heartbeat_front_addr":"172.21.15.183:6821/2743896476","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.763716+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.782386+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.712986+0000","dead_epoch":39},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.846199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.735683+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.776861+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.767124+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T08:55:51.746641+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":6}]},{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T08:59:04.164 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 msgr2=0x7f290c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 0x7f290c06e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2924009b40 tx=0x7f2924009190 comp rx=0 tx=0).stop 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 msgr2=0x7f2928159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2920002050 tx=0x7f29200015b0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 shutdown_connections 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f290c06be30 0x7f290c06e2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.165 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f2928153e40 0x7f2928159ed0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2928152380 0x7f2928153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 --2- 172.21.15.49:0/962462325 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f29140051c0 0x7f29280c2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 >> 172.21.15.49:0/962462325 conn(0x7f2928074840 msgr2=0x7f2928151020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:04.166 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 shutdown_connections 2024-05-09T08:59:04.167 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.164+0000 7f293002c640 1 -- 172.21.15.49:0/962462325 wait complete. 2024-05-09T08:59:04.175 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T08:59:04.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.281+0000 7fd2c7cae640 1 Processor -- start 2024-05-09T08:59:04.280 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.281+0000 7fd2c7cae640 1 -- start start 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c7cae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2c006c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c7cae640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c0158000 con 0x7fd2c01512d0 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c7cae640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c0157480 con 0x7fd2c015ac60 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c7cae640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c006c900 con 0x7fd2c006bf80 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c5a4a640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fd2c01512d0 0x7fd2c006b4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41132/0 (socket says 172.21.15.49:41132) 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c5a4a640 1 -- 172.21.15.49:0/3674606406 learned_addr learned my addr 172.21.15.49:0/3674606406 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:04.283 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c5249640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2c006c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756712486 0 0) 0x7fd2c0158000 con 0x7fd2c01512d0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.282+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2ac003520 con 0x7fd2c01512d0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2500834681 0 0) 0x7fd2ac003520 con 0x7fd2c01512d0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 msgr2=0x7fd2c006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2c006c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 >> v1:172.21.15.183:6789/0 conn(0x7fd2c015ac60 legacy=0x7fd2c0161a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0157760 con 0x7fd2c01512d0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fd2bc001670 con 0x7fd2c01512d0 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 >> v1:172.21.15.49:6789/0 conn(0x7fd2c01512d0 legacy=0x7fd2c006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2ac0039e0 0x7fd2ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.284 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2ac004d40 0x7fd2ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac00bb00 con 0x7fd2ac0039e0 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac00bc80 con 0x7fd2ac004d40 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac00be80 con 0x7fd2c006bf80 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2ac0039e0 0x7fd2ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5249640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c624b640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2ac004d40 0x7fd2ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 -- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 msgr2=0x7fd2ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2c006bf80 0x7fd2ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 -- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2ac004d40 msgr2=0x7fd2ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2ac004d40 0x7fd2ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.283+0000 7fd2c5a4a640 1 -- 172.21.15.49:0/3674606406 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac003520 con 0x7fd2ac0039e0 2024-05-09T08:59:04.285 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2ac0039e0 0x7fd2ac003e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c210 tx=0x7fd2ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75ba29c64b5e002 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007cf0 con 0x7fd2ac0039e0 2024-05-09T08:59:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd2bc005af0 con 0x7fd2ac0039e0 2024-05-09T08:59:04.286 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c4a48640 1 -- 172.21.15.49:0/3674606406 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007070 con 0x7fd2ac0039e0 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c7cae640 1 -- 172.21.15.49:0/3674606406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2ac0039e0 msgr2=0x7fd2ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/3674606406 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2ac0039e0 0x7fd2ac003e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c210 tx=0x7fd2ac005830 comp rx=0 tx=0).stop 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.284+0000 7fd2c624b640 1 -- 172.21.15.49:0/3674606406 reap_dead start 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.285+0000 7fd2c7cae640 1 -- 172.21.15.49:0/3674606406 shutdown_connections 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.285+0000 7fd2c7cae640 1 -- 172.21.15.49:0/3674606406 >> 172.21.15.49:0/3674606406 conn(0x7fd2c00cc290 msgr2=0x7fd2c00cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.285+0000 7fd2c7cae640 1 -- 172.21.15.49:0/3674606406 shutdown_connections 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.285+0000 7fd2c7cae640 1 -- 172.21.15.49:0/3674606406 wait complete. 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 Processor -- start 2024-05-09T08:59:04.289 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 -- start start 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2ac0039e0 0x7fd2c014d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c014f5d0 0x7fd2c00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c00c8af0 con 0x7fd2c01519a0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.286+0000 7fd2c7cae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c0157d20 con 0x7fd2c014f5d0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c7cae640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c01570b0 con 0x7fd2ac0039e0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35098/0 (socket says 172.21.15.49:35098) 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 -- 172.21.15.49:0/1377052997 learned_addr learned my addr 172.21.15.49:0/1377052997 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2ac0039e0 0x7fd2c014d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c624b640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c014f5d0 0x7fd2c00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 -- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2ac0039e0 msgr2=0x7fd2c014d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.290 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2ac0039e0 0x7fd2c014d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 -- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c014f5d0 msgr2=0x7fd2c00cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c014f5d0 0x7fd2c00cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 -- 172.21.15.49:0/1377052997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c01ed200 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c5249640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011a30 tx=0x7fd2b0011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000b040 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fd2b000fd90 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001c6c0 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c01502c0 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.287+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c0150860 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.288+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c004f660 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.289+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2b001c860 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.289+0000 7fd2b67fc640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 0x7fd2a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.289+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5886+0+0 (secure 0 0 0) 0x7fd2b008ba80 con 0x7fd2c01519a0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.289+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 0x7fd2a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.291 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.290+0000 7fd2c5a4a640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 0x7fd2a406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc032000 tx=0x7fd2bc034040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.292 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.294+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd2b0059d30 con 0x7fd2c01519a0 2024-05-09T08:59:04.518 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.519+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd2c0155a60 con 0x7fd2c01519a0 2024-05-09T08:59:04.520 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.521+0000 7fd2b67fc640 1 -- 172.21.15.49:0/1377052997 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7fd2b005e030 con 0x7fd2c01519a0 2024-05-09T08:59:04.520 INFO:teuthology.orchestra.run.smithi049.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.524+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 msgr2=0x7fd2a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.524+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 0x7fd2a406e2f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc032000 tx=0x7fd2bc034040 comp rx=0 tx=0).stop 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.524+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 msgr2=0x7fd2c014f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.524+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0011a30 tx=0x7fd2b0011ef0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.524+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 shutdown_connections 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fd2a406be30 0x7fd2a406e2f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2c014f5d0 0x7fd2c00cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.524 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fd2c01519a0 0x7fd2c014f090 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 --2- 172.21.15.49:0/1377052997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd2ac0039e0 0x7fd2c014d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 >> 172.21.15.49:0/1377052997 conn(0x7fd2c00cc290 msgr2=0x7fd2c00c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:04.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 shutdown_connections 2024-05-09T08:59:04.525 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.525+0000 7fd2c7cae640 1 -- 172.21.15.49:0/1377052997 wait complete. 2024-05-09T08:59:04.528 INFO:tasks.ceph.mds.c.smithi049.stderr: -5626> 2024-05-09T08:56:17.723+0000 7fe0b0331640 -1 quiesce.mds.1 failed to decode message of type 1285 v1: End of buffer [buffer:2] 2024-05-09T08:59:04.538 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T08:59:04.562 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2024-05-09T08:59:04.562 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2024-05-09T08:59:04.569 INFO:tasks.ceph.mds.e.smithi049.stdout:starting mds.e at 2024-05-09T08:59:04.572 INFO:tasks.ceph.mds.a.smithi049.stdout:starting mds.a at 2024-05-09T08:59:04.575 INFO:tasks.ceph.mds.c.smithi049.stdout:starting mds.c at 2024-05-09T08:59:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.649+0000 7f480c129640 1 Processor -- start 2024-05-09T08:59:04.649 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.649+0000 7f480c129640 1 -- start start 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480c129640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4804154b90 0x7f48040cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480c129640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48040c6d50 0x7f48040cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480c129640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4804072600 con 0x7f4804154f70 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480c129640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f48040728e0 con 0x7f48040c6d50 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480c129640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7f48040c5880 con 0x7f4804154b90 2024-05-09T08:59:04.650 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f480a6c6640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f48040c6d50 0x7f48040cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f48096c4640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f4804154f70 0x7f48040c6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41178/0 (socket says 172.21.15.49:41178) 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f48096c4640 1 -- 172.21.15.49:0/3811164399 learned_addr learned my addr 172.21.15.49:0/3811164399 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.650+0000 7f4809ec5640 1 --2- 172.21.15.49:0/3811164399 >> v2:172.21.15.183:3300/0 conn(0x7f4804154b90 0x7f48040cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f480a6c6640 1 -- 172.21.15.49:0/3811164399 >> v2:172.21.15.183:3300/0 conn(0x7f4804154b90 msgr2=0x7f48040cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f480a6c6640 1 --2- 172.21.15.49:0/3811164399 >> v2:172.21.15.183:3300/0 conn(0x7f4804154b90 0x7f48040cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f480a6c6640 1 -- 172.21.15.49:0/3811164399 >> v1:172.21.15.49:6789/0 conn(0x7f4804154f70 legacy=0x7f48040c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f4808ec3640 1 -- 172.21.15.49:0/3811164399 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3300020020 0 0) 0x7f4804072600 con 0x7f4804154f70 2024-05-09T08:59:04.651 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f480a6c6640 1 -- 172.21.15.49:0/3811164399 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48040c5b60 con 0x7f48040c6d50 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f480a6c6640 1 --2- 172.21.15.49:0/3811164399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48040c6d50 0x7f48040cb7c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f48040cdf50 tx=0x7f47fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3da1d6d1a1e0e2e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f4808ec3640 1 -- 172.21.15.49:0/3811164399 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc007740 con 0x7f48040c6d50 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.651+0000 7f4808ec3640 1 -- 172.21.15.49:0/3811164399 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f47fc0078e0 con 0x7f48040c6d50 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f4808ec3640 1 -- 172.21.15.49:0/3811164399 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc007cd0 con 0x7f48040c6d50 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 -- 172.21.15.49:0/3811164399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48040c6d50 msgr2=0x7f48040cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 --2- 172.21.15.49:0/3811164399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48040c6d50 0x7f48040cb7c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f48040cdf50 tx=0x7f47fc02a710 comp rx=0 tx=0).stop 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 -- 172.21.15.49:0/3811164399 shutdown_connections 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 --2- 172.21.15.49:0/3811164399 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f48040c6d50 0x7f48040cb7c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 --2- 172.21.15.49:0/3811164399 >> v2:172.21.15.183:3300/0 conn(0x7f4804154b90 0x7f48040cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 -- 172.21.15.49:0/3811164399 >> 172.21.15.49:0/3811164399 conn(0x7f4804074840 msgr2=0x7f4804074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:04.653 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 -- 172.21.15.49:0/3811164399 shutdown_connections 2024-05-09T08:59:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.652+0000 7f480c129640 1 -- 172.21.15.49:0/3811164399 wait complete. 2024-05-09T08:59:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.653+0000 7f480c129640 1 Processor -- start 2024-05-09T08:59:04.654 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.653+0000 7f480c129640 1 -- start start 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48040c6d50 0x7f48040d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4804154f70 0x7f48040cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f480415af40 con 0x7f4804154b90 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48040c6120 con 0x7f48040c6d50 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480c129640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f4804071820 con 0x7f4804154f70 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35162/0 (socket says 172.21.15.49:35162) 2024-05-09T08:59:04.656 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 -- 172.21.15.49:0/1762851441 learned_addr learned my addr 172.21.15.49:0/1762851441 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f480a6c6640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4804154f70 0x7f48040cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f4809ec5640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48040c6d50 0x7f48040d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 -- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4804154f70 msgr2=0x7f48040cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4804154f70 0x7f48040cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 -- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48040c6d50 msgr2=0x7f48040d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48040c6d50 0x7f48040d02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.654+0000 7f48096c4640 1 -- 172.21.15.49:0/1762851441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041f4580 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f48096c4640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f480000c430 tx=0x7f480000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000074e0 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f48000047e0 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800004bf0 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48040d0e60 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.655+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48040cef40 con 0x7f4804154b90 2024-05-09T08:59:04.657 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.656+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48040d1380 con 0x7f4804154b90 2024-05-09T08:59:04.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.656+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4800011070 con 0x7f4804154b90 2024-05-09T08:59:04.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.657+0000 7f47f67fc640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 0x7f47e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:04.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.657+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6316+0+0 (secure 0 0 0) 0x7f480008ae40 con 0x7f4804154b90 2024-05-09T08:59:04.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.657+0000 7f4809ec5640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 0x7f47e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:04.658 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.657+0000 7f4809ec5640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 0x7f47e806e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006fd0 tx=0x7f47f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:04.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.661+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f480000e280 con 0x7f4804154b90 2024-05-09T08:59:04.882 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:04.883+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f480404f660 con 0x7f4804154b90 2024-05-09T08:59:05.526 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.526+0000 7f47f67fc640 1 -- 172.21.15.49:0/1762851441 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f4800059500 con 0x7f4804154b90 2024-05-09T08:59:05.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 msgr2=0x7f47e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 0x7f47e806e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006fd0 tx=0x7f47f8008040 comp rx=0 tx=0).stop 2024-05-09T08:59:05.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 msgr2=0x7f48041f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f480000c430 tx=0x7f480000c8f0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.529 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 shutdown_connections 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f47e806be30 0x7f47e806e2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4804154f70 0x7f48040cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f4804154b90 0x7f48041f3bc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 --2- 172.21.15.49:0/1762851441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48040c6d50 0x7f48040d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.529+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 >> 172.21.15.49:0/1762851441 conn(0x7f4804074840 msgr2=0x7f48040cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.530+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 shutdown_connections 2024-05-09T08:59:05.530 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.530+0000 7f480c129640 1 -- 172.21.15.49:0/1762851441 wait complete. 2024-05-09T08:59:05.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T08:59:05.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.698+0000 7f5c4f99b640 1 Processor -- start 2024-05-09T08:59:05.697 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.698+0000 7f5c4f99b640 1 -- start start 2024-05-09T08:59:05.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4f99b640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c48154f30 0x7f5c4815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:05.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4f99b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c480cc9c0 con 0x7f5c480cb3c0 2024-05-09T08:59:05.698 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4f99b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c480ccc60 con 0x7f5c4815ef70 2024-05-09T08:59:05.699 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4f99b640 1 -- --> v2:172.21.15.49:3300/0 -- mon_getmap magic: 0 -- 0x7f5c4815bc70 con 0x7f5c48154f30 2024-05-09T08:59:05.700 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4d737640 1 --2- >> v2:172.21.15.49:3300/0 conn(0x7f5c48154f30 0x7f5c4815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4cf36640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7f5c480cb3c0 0x7f5c480c5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41184/0 (socket says 172.21.15.49:41184) 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.699+0000 7f5c4cf36640 1 -- 172.21.15.49:0/21436313 learned_addr learned my addr 172.21.15.49:0/21436313 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c4d737640 1 -- 172.21.15.49:0/21436313 >> v1:172.21.15.183:6789/0 conn(0x7f5c4815ef70 legacy=0x7f5c48074d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c4d737640 1 -- 172.21.15.49:0/21436313 >> v1:172.21.15.49:6789/0 conn(0x7f5c480cb3c0 legacy=0x7f5c480c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c4d737640 1 -- 172.21.15.49:0/21436313 --> v2:172.21.15.49:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4815e490 con 0x7f5c48154f30 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c3ffff640 1 -- 172.21.15.49:0/21436313 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162662994 0 0) 0x7f5c480cc9c0 con 0x7f5c480cb3c0 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c4d737640 1 --2- 172.21.15.49:0/21436313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c48154f30 0x7f5c4815b730 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097a0 tx=0x7f5c4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f176697f66fab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c3ffff640 1 -- 172.21.15.49:0/21436313 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037070 con 0x7f5c48154f30 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c3ffff640 1 -- 172.21.15.49:0/21436313 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5c44032440 con 0x7f5c48154f30 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.700+0000 7f5c3ffff640 1 -- 172.21.15.49:0/21436313 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44032830 con 0x7f5c48154f30 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 -- 172.21.15.49:0/21436313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c48154f30 msgr2=0x7f5c4815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.701 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/21436313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c48154f30 0x7f5c4815b730 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097a0 tx=0x7f5c4402a230 comp rx=0 tx=0).stop 2024-05-09T08:59:05.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 -- 172.21.15.49:0/21436313 shutdown_connections 2024-05-09T08:59:05.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/21436313 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c48154f30 0x7f5c4815b730 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 -- 172.21.15.49:0/21436313 >> 172.21.15.49:0/21436313 conn(0x7f5c480d0960 msgr2=0x7f5c480d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:05.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 -- 172.21.15.49:0/21436313 shutdown_connections 2024-05-09T08:59:05.702 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.701+0000 7f5c4f99b640 1 -- 172.21.15.49:0/21436313 wait complete. 2024-05-09T08:59:05.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.702+0000 7f5c4f99b640 1 Processor -- start 2024-05-09T08:59:05.705 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.702+0000 7f5c4f99b640 1 -- start start 2024-05-09T08:59:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480cb3c0 0x7f5c481534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c48154f30 0x7f5c481f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4815ea50 con 0x7f5c4815ef70 2024-05-09T08:59:05.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4815e240 con 0x7f5c480cb3c0 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4f99b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7f5c480cbc30 con 0x7f5c48154f30 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35176/0 (socket says 172.21.15.49:35176) 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 -- 172.21.15.49:0/886936450 learned_addr learned my addr 172.21.15.49:0/886936450 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4d737640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480cb3c0 0x7f5c481534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4cf36640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c48154f30 0x7f5c481f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:05.707 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 -- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c48154f30 msgr2=0x7f5c481f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c48154f30 0x7f5c481f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 -- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480cb3c0 msgr2=0x7f5c481534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.703+0000 7f5c4df38640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480cb3c0 0x7f5c481534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:05.708 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c4df38640 1 -- 172.21.15.49:0/886936450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c48153bb0 con 0x7f5c4815ef70 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c4df38640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c6b0 tx=0x7f5c4000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40003b70 con 0x7f5c4815ef70 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5c40003d30 con 0x7f5c4815ef70 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c48153e90 con 0x7f5c4815ef70 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40005330 con 0x7f5c4815ef70 2024-05-09T08:59:05.709 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.704+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481f4890 con 0x7f5c4815ef70 2024-05-09T08:59:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.705+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c480cec80 con 0x7f5c4815ef70 2024-05-09T08:59:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.706+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c400054d0 con 0x7f5c4815ef70 2024-05-09T08:59:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.706+0000 7f5c3dffb640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.706+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6316+0+0 (secure 0 0 0) 0x7f5c4008af70 con 0x7f5c4815ef70 2024-05-09T08:59:05.710 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.706+0000 7f5c4d737640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:05.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.707+0000 7f5c4d737640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c44034000 tx=0x7f5c4402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:05.711 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.710+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5c480cec80 con 0x7f5c4815ef70 2024-05-09T08:59:05.891 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:05.892+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5c48068c70 con 0x7f5c4815ef70 2024-05-09T08:59:06.528 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.528+0000 7f5c3dffb640 1 -- 172.21.15.49:0/886936450 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f5c40058f40 con 0x7f5c4815ef70 2024-05-09T08:59:06.528 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_metadata' removed 2024-05-09T08:59:06.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.534+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 msgr2=0x7f5c2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.534+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c44034000 tx=0x7f5c4402e000 comp rx=0 tx=0).stop 2024-05-09T08:59:06.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.534+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 msgr2=0x7f5c4815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.534+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c6b0 tx=0x7f5c4000cb70 comp rx=0 tx=0).stop 2024-05-09T08:59:06.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 shutdown_connections 2024-05-09T08:59:06.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7f5c2c06be30 0x7f5c2c06e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7f5c4815ef70 0x7f5c4815a030 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c48154f30 0x7f5c481f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.534 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 --2- 172.21.15.49:0/886936450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c480cb3c0 0x7f5c481534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 >> 172.21.15.49:0/886936450 conn(0x7f5c480d0960 msgr2=0x7f5c480ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:06.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 shutdown_connections 2024-05-09T08:59:06.535 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.535+0000 7f5c4f99b640 1 -- 172.21.15.49:0/886936450 wait complete. 2024-05-09T08:59:06.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T08:59:06.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.663+0000 7fe7e651b640 1 Processor -- start 2024-05-09T08:59:06.663 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e651b640 1 -- start start 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e651b640 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7e0154ed0 0x7fe7e0072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e651b640 1 -- --> v1:172.21.15.49:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e01552b0 con 0x7fe7e006b200 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e651b640 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e015ade0 con 0x7fe7e00c8fe0 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e651b640 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e006d280 con 0x7fe7e0154ed0 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.664+0000 7fe7e4ab8640 1 --1- >> v1:172.21.15.49:6789/0 conn(0x7fe7e006b200 0x7fe7e0165760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.49:6789/0 says I am v1:172.21.15.49:41196/0 (socket says 172.21.15.49:41196) 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7e4ab8640 1 -- 172.21.15.49:0/481815412 learned_addr learned my addr 172.21.15.49:0/481815412 (peer_addr_for_me v1:172.21.15.49:0/0) 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v1:172.21.15.49:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906075826 0 0) 0x7fe7e01552b0 con 0x7fe7e006b200 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 --> v1:172.21.15.49:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7cc003520 con 0x7fe7e006b200 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 --2- 172.21.15.49:0/481815412 >> v2:172.21.15.183:3300/0 conn(0x7fe7e0154ed0 0x7fe7e0072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v1:172.21.15.49:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2678693345 0 0) 0x7fe7cc003520 con 0x7fe7e006b200 2024-05-09T08:59:06.667 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 >> v2:172.21.15.183:3300/0 conn(0x7fe7e0154ed0 msgr2=0x7fe7e0072a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 --2- 172.21.15.49:0/481815412 >> v2:172.21.15.183:3300/0 conn(0x7fe7e0154ed0 0x7fe7e0072a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 >> v1:172.21.15.183:6789/0 conn(0x7fe7e00c8fe0 legacy=0x7fe7e015e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 --> v1:172.21.15.49:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e0073400 con 0x7fe7e006b200 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v1:172.21.15.49:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3551097152 0 0) 0x7fe7d80019f0 con 0x7fe7e006b200 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 >> v1:172.21.15.49:6789/0 conn(0x7fe7e006b200 legacy=0x7fe7e0165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7cc004800 0x7fe7cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc0051c0 0x7fe7cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc00c030 con 0x7fe7cc0039b0 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc00c1e0 con 0x7fe7cc0051c0 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe7cc00c3e0 con 0x7fe7cc004800 2024-05-09T08:59:06.668 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7d7fff640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7cc004800 0x7fe7cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7e4ab8640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc0051c0 0x7fe7cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 -- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7cc004800 msgr2=0x7fe7cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7cc004800 0x7fe7cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 -- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc0051c0 msgr2=0x7fe7cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc0051c0 0x7fe7cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7dffff640 1 -- 172.21.15.49:0/481815412 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc003520 con 0x7fe7cc0039b0 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.665+0000 7fe7e4ab8640 1 -- 172.21.15.49:0/481815412 reap_dead start 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.666+0000 7fe7dffff640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7cc003dd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009ac0 tx=0x7fe7cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e967263dc06647d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:06.669 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.666+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0036070 con 0x7fe7cc0039b0 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.666+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe7d0031400 con 0x7fe7cc0039b0 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.666+0000 7fe7df7fe640 1 -- 172.21.15.49:0/481815412 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0009690 con 0x7fe7cc0039b0 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 -- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 msgr2=0x7fe7cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7cc003dd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009ac0 tx=0x7fe7cc005830 comp rx=0 tx=0).stop 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 -- 172.21.15.49:0/481815412 shutdown_connections 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 --2- 172.21.15.49:0/481815412 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7cc003dd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 -- 172.21.15.49:0/481815412 >> 172.21.15.49:0/481815412 conn(0x7fe7e0067d90 msgr2=0x7fe7e0074840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 -- 172.21.15.49:0/481815412 shutdown_connections 2024-05-09T08:59:06.670 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.667+0000 7fe7e651b640 1 -- 172.21.15.49:0/481815412 wait complete. 2024-05-09T08:59:06.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.668+0000 7fe7e651b640 1 Processor -- start 2024-05-09T08:59:06.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.668+0000 7fe7e651b640 1 -- start start 2024-05-09T08:59:06.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.673 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 0x7fe7e01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 0x7fe7e01e1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 -- --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e015b660 con 0x7fe7cc0039b0 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e015b380 con 0x7fe7e01e1480 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e651b640 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e015b940 con 0x7fe7e0158530 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 --2- >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.49:3300/0 says I am v2:172.21.15.49:35196/0 (socket says 172.21.15.49:35196) 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 -- 172.21.15.49:0/2338954904 learned_addr learned my addr 172.21.15.49:0/2338954904 (peer_addr_for_me v2:172.21.15.49:0/0) 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7d7fff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 0x7fe7e01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.674 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e4ab8640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 0x7fe7e01e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 -- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 msgr2=0x7fe7e01560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 0x7fe7e01560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 -- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 msgr2=0x7fe7e01e1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 0x7fe7e01e1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7dffff640 1 -- 172.21.15.49:0/2338954904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e01e1e40 con 0x7fe7cc0039b0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7e4ab8640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 0x7fe7e01e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.669+0000 7fe7d7fff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 0x7fe7e01560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7dffff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009be0 tx=0x7fe7d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d003f020 con 0x7fe7cc0039b0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe7d0037040 con 0x7fe7cc0039b0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e0156fb0 con 0x7fe7cc0039b0 2024-05-09T08:59:06.675 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0036070 con 0x7fe7cc0039b0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.670+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e0157570 con 0x7fe7cc0039b0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.671+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e0068560 con 0x7fe7cc0039b0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.671+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7d0042020 con 0x7fe7cc0039b0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.672+0000 7fe7dd7fa640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.672+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5878+0+0 (secure 0 0 0) 0x7fe7d00aafd0 con 0x7fe7cc0039b0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.672+0000 7fe7d7fff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.673+0000 7fe7d7fff640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 0x7fe7c006e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T08:59:06.676 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.676+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe7d0049030 con 0x7fe7cc0039b0 2024-05-09T08:59:06.864 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:06.865+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 --> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7e00737d0 con 0x7fe7cc0039b0 2024-05-09T08:59:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.534+0000 7fe7dd7fa640 1 -- 172.21.15.49:0/2338954904 <== mon.0 v2:172.21.15.49:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7fe7e01e1e40 con 0x7fe7cc0039b0 2024-05-09T08:59:07.533 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cephfs_data' removed 2024-05-09T08:59:07.537 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.537+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 msgr2=0x7fe7c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.537+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 0x7fe7c006e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000c00 tx=0x7fe7c8009040 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.537+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 msgr2=0x7fe7e00c7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.537+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009be0 tx=0x7fe7d0004730 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.537+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 shutdown_connections 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:6832/3271117140,v1:172.21.15.183:6833/3271117140] conn(0x7fe7c006be30 0x7fe7c006e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7e01e1480 0x7fe7e01e1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe7e0158530 0x7fe7e01560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 --2- 172.21.15.49:0/2338954904 >> [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] conn(0x7fe7cc0039b0 0x7fe7e00c7e20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 >> 172.21.15.49:0/2338954904 conn(0x7fe7e0067d90 msgr2=0x7fe7e00cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 shutdown_connections 2024-05-09T08:59:07.538 INFO:teuthology.orchestra.run.smithi049.stderr:2024-05-09T08:59:07.538+0000 7fe7e651b640 1 -- 172.21.15.49:0/2338954904 wait complete. 2024-05-09T08:59:07.552 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T08:59:07.552 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T08:59:07.552 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.562 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T08:59:07.562 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T08:59:07.562 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.612 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T08:59:07.613 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-05-09T08:59:07.613 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.663 INFO:tasks.ceph.mds.e:Stopped 2024-05-09T08:59:07.664 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T08:59:07.664 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.675 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T08:59:07.676 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T08:59:07.676 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.728 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T08:59:07.728 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-05-09T08:59:07.728 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.780 INFO:tasks.ceph.mds.f:Stopped 2024-05-09T08:59:07.780 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T08:59:07.781 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T08:59:07.781 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.813 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T08:59:07.813 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T08:59:07.813 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.883 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T08:59:07.883 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T08:59:07.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:07.953 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T08:59:07.955 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T08:59:07.955 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.020 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T08:59:08.020 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T08:59:08.020 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.053 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T08:59:08.054 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T08:59:08.054 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.132 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T08:59:08.133 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T08:59:08.133 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.205 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T08:59:08.205 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T08:59:08.205 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.284 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T08:59:08.284 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T08:59:08.284 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T08:59:08.284 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.311 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T08:59:08.312 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T08:59:08.312 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.354 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T08:59:08.355 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T08:59:08.355 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T08:59:08.355 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.368 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T08:59:08.369 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T08:59:08.369 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.407 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T08:59:08.407 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T08:59:08.408 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T08:59:08.460 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T08:59:08.461 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T08:59:08.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-05-09T08:59:08.508 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:59:08.508 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T08:59:08.744 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:59:08.744 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T08:59:08.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:59:08.858 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T08:59:08.940 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:59:08.940 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T08:59:08.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:59:08.999 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T08:59:09.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:59:09.250 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T08:59:09.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:59:09.305 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T08:59:09.354 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:59:09.354 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T08:59:09.411 INFO:tasks.ceph:Archiving mon data... 2024-05-09T08:59:09.412 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/data/mon.a.tgz 2024-05-09T08:59:09.412 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-05-09T08:59:09.432 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.j3JjcYciGQ 2024-05-09T08:59:09.432 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.j3JjcYciGQ 2024-05-09T08:59:09.584 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.j3JjcYciGQ 2024-05-09T08:59:09.670 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.j3JjcYciGQ is 352KB 2024-05-09T08:59:09.765 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.j3JjcYciGQ 2024-05-09T08:59:09.782 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/data/mon.b.tgz 2024-05-09T08:59:09.782 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-05-09T08:59:09.798 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.DpOTqyLm7M 2024-05-09T08:59:09.799 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DpOTqyLm7M 2024-05-09T08:59:09.993 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.DpOTqyLm7M 2024-05-09T08:59:10.075 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.DpOTqyLm7M is 488KB 2024-05-09T08:59:10.145 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.DpOTqyLm7M 2024-05-09T08:59:10.159 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/data/mon.c.tgz 2024-05-09T08:59:10.159 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-05-09T08:59:10.213 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.KL9sP3LuXZ 2024-05-09T08:59:10.213 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.KL9sP3LuXZ 2024-05-09T08:59:10.366 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.KL9sP3LuXZ 2024-05-09T08:59:10.447 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.KL9sP3LuXZ is 488KB 2024-05-09T08:59:10.544 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.KL9sP3LuXZ 2024-05-09T08:59:10.558 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T08:59:10.558 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T08:59:10.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T08:59:10.648 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T08:59:10.675 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T08:59:10.675 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T08:59:10.677 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi049/crash 2024-05-09T08:59:10.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T08:59:10.717 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi183/crash 2024-05-09T08:59:10.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T08:59:10.740 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T08:59:10.741 DEBUG:teuthology.orchestra.run.smithi049:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:59:10.762 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T08:59:10.843 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T08:59:10.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T08:59:10.895 INFO:tasks.ceph:Compressing logs... 2024-05-09T08:59:10.896 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:59:10.916 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T08:59:10.941 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59683.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41442.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47302.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59683.log.gz 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47302.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41442.log.gz 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41619.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44940.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47302.log.gz 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41619.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58049.log 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41619.log.gz 2024-05-09T08:59:10.944 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47101.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5/var/log/ceph/ceph.log: --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36288.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T08:59:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36288.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36288.log.gz 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36419.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mgr.x.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36208.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5/var/log/ceph/ceph-mon.c.log: --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-09T08:59:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36248.log 2024-05-09T08:59:10.971 INFO:teuthology.orchestra.run.smithi183.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36208.log.gz 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36379.log 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36248.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36248.log.gz 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36337.log 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36379.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36379.log.gz 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36459.log 2024-05-09T08:59:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36337.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36337.log.gz 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36499.log 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36459.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36459.log.gz 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.36499.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36499.log.gz 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T08:59:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T08:59:11.011 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T08:59:11.031 INFO:teuthology.orchestra.run.smithi183.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61402.log 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44940.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59566.log 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58049.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47101.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60871.log 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59566.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61402.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59566.log.gz 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-05-09T08:59:11.066 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60871.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59342.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:gzip/var/log/ceph/ceph-client.admin.47769.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59033.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55471.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59033.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59342.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59033.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.40824.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55471.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46702.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-05-09T08:59:11.067 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61670.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46702.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58852.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61670.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40970.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.40970.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58852.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56925.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40970.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55754.log 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2024-05-09T08:59:11.068 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59031.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58306.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56925.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55754.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48078.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59031.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58306.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47645.log 87.2% 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48078.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47262.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60144.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-05-09T08:59:11.069 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60144.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47645.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7%/var/log/ceph/ceph-client.admin.46275.log: -- replaced with /var/log/ceph/ceph-client.admin.60144.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:gzip/var/log/ceph/ceph-client.admin.47707.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40924.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48740.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41316.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.40924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41661.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40924.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41316.log.gz 2024-05-09T08:59:11.070 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43379.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41661.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49330.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43379.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57097.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55399.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57097.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42342.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57097.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44461.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55399.log: /var/log/ceph/ceph-client.admin.42342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47789.log 2024-05-09T08:59:11.071 INFO:teuthology.orchestra.run.smithi049.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55399.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.44461.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42342.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60619.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44461.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54627.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47789.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60619.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54627.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54627.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.2.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44938.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-05-09T08:59:11.072 INFO:teuthology.orchestra.run.smithi049.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59986.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44938.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54131.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42693.log: /var/log/ceph/ceph-client.admin.59986.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58684.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59986.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54131.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58684.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54131.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56112.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.audit.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58684.log.gz 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46520.log 2024-05-09T08:59:11.073 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46520.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.56112.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61057.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46520.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61353.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55578.log: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60829.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61057.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61353.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61057.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52622.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61353.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60829.log.gz 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2024-05-09T08:59:11.074 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43421.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52622.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43757.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59505.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41577.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59505.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43421.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59505.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61267.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41577.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60787.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2024-05-09T08:59:11.075 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60787.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61267.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58516.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60787.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59032.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58516.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59032.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58516.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44021.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59032.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59029.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43211.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60955.log 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44021.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59029.log.gz 2024-05-09T08:59:11.076 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53779.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43211.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42010.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60955.log: /var/log/ceph/ceph-client.admin.53779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60955.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53779.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42010.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56585.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59340.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45980.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-05-09T08:59:11.077 INFO:teuthology.orchestra.run.smithi049.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.56585.log.gz/var/log/ceph/ceph-client.admin.59340.log: 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45980.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49326.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59340.log.gz 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45980.log.gz 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44698.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58180.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/objectstore_tool.49326.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48160.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59345.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48160.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58180.log.gz 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42698.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59345.log: gzip -5 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48160.log.gz 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53853.log 2024-05-09T08:59:11.078 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59345.log.gz/var/log/ceph/ceph-client.admin.42698.log: 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42698.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58390.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53853.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60493.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58390.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58390.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59681.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log/var/log/ceph/ceph-client.admin.60493.log: 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59681.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60493.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr: /var/log/ceph/ceph-client.admin.60228.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59681.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59682.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.60228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60102.log 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59682.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60228.log.gz 2024-05-09T08:59:11.079 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59682.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48915.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60102.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48915.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60102.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.49326.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47461.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59685.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47461.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43955.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55971.log 2024-05-09T08:59:11.080 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54111.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59685.log: /var/log/ceph/ceph-client.admin.55971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55876.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59685.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55971.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.54111.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55557.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.55876.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43253.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56682.log 2024-05-09T08:59:11.081 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55557.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55557.log.gz 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60535.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56682.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.51103.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56682.log.gz 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60535.log.gz/var/log/ceph/objectstore_tool.51103.log: 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54842.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61539.log: /var/log/ceph/ceph-client.admin.51064.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48459.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54842.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.54842.log.gz 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-05-09T08:59:11.082 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41787.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48459.log: /var/log/ceph/ceph-client.admin.47198.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.48459.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41829.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41787.log: /var/log/ceph/ceph-client.admin.45678.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41787.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52889.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.51103.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41829.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41829.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42005.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58091.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42005.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52889.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42005.log.gz 2024-05-09T08:59:11.083 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59034.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58091.log: gzip/var/log/ceph/ceph-client.admin.47979.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58432.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59034.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58091.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59034.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58768.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41178.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4%/var/log/ceph/ceph-client.admin.58768.log: -- replaced with /var/log/ceph/ceph-client.admin.58432.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55224.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58768.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60703.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41178.log: /var/log/ceph/ceph-client.admin.55224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44802.log 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60703.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41178.log.gz 2024-05-09T08:59:11.084 INFO:teuthology.orchestra.run.smithi049.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55224.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53402.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60703.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44802.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.44802.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56228.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7%/var/log/ceph/ceph-client.admin.56228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42008.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53402.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56228.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53165.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29329.log/var/log/ceph/ceph-client.admin.42008.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mon.a.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42944.log 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42008.log.gz 2024-05-09T08:59:11.085 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.tmp-client.admin.29329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29329.log.gz 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59030.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42944.log: /var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60913.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59030.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42944.log.gz 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59030.log.gz 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41358.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60913.log.gz 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40709.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz/var/log/ceph/ceph-client.admin.41358.log: 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42902.log 2024-05-09T08:59:11.086 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41358.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61141.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.40709.log: /var/log/ceph/ceph-client.admin.42902.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40709.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42902.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43337.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41086.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43337.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61141.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60409.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59944.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60997.log 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60409.log: 87.5%/var/log/ceph/ceph-client.admin.59944.log: -- replaced with /var/log/ceph/ceph-client.admin.41086.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60409.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59944.log.gz 2024-05-09T08:59:11.087 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59346.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59706.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1%/var/log/ceph/ceph-client.admin.59346.log: -- replaced with /var/log/ceph/ceph-client.admin.60997.log.gz 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42340.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59346.log.gz 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.59706.log: /var/log/ceph/ceph-client.admin.45321.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42340.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59706.log.gzgzip 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61487.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5/var/log/ceph/ceph-client.admin.45321.log: --verbose -- /var/log/ceph/ceph-client.admin.43631.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45321.log.gz 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58264.log 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61487.log.gz 2024-05-09T08:59:11.088 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43631.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43631.log.gz 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54430.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42699.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5%/var/log/ceph/ceph-client.admin.54430.log: -- replaced with /var/log/ceph/ceph-client.admin.58264.log.gz 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58642.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59347.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42699.log: /var/log/ceph/ceph-client.admin.58642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59347.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42699.log.gz 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58642.log.gz 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59347.log.gz 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5/var/log/ceph/ceph-client.admin.47916.log: --verbose -- /var/log/ceph/ceph-client.admin.60186.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61628.log 2024-05-09T08:59:11.089 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50179.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60186.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.61628.log: -- /var/log/ceph/ceph-client.admin.59024.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/objectstore_tool.50179.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61628.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60186.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41484.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53636.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59024.log: /var/log/ceph/ceph-client.admin.41484.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59024.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41484.log.gz 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60451.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61183.log 2024-05-09T08:59:11.090 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60451.log: /var/log/ceph/ceph-client.admin.53636.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53636.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60451.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48257.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42334.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61183.log: /var/log/ceph/ceph-client.admin.48257.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61183.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48257.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61099.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58726.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61099.log: /var/log/ceph/ceph-client.admin.42334.log: 88.4% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42334.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61099.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41222.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46722.log 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58726.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.50179.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41222.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58726.log.gz 2024-05-09T08:59:11.091 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41222.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: -- /var/log/ceph/ceph-client.admin.58138.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53955.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46722.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42986.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58138.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35783.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42986.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.53955.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61444.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42986.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45540.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.35783.log: gzip -5/var/log/ceph/ceph-client.admin.61444.log: --verbose -- /var/log/ceph/ceph-client.admin.54485.log 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45540.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61444.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35783.log.gz 2024-05-09T08:59:11.092 INFO:teuthology.orchestra.run.smithi049.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.45540.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4%/var/log/ceph/ceph-mds.e.log: -- replaced with /var/log/ceph/ceph-client.admin.54485.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60060.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49255.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47894.log: /var/log/ceph/ceph-client.admin.60060.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60060.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49255.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49255.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60661.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42561.log: /var/log/ceph/ceph-client.admin.43124.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47057.log 2024-05-09T08:59:11.093 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47057.log: /var/log/ceph/ceph-client.admin.60661.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60661.log.gz 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47057.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41266.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42345.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41266.log: /var/log/ceph/ceph-client.admin.44638.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41266.log.gz 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59684.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60577.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42345.log: /var/log/ceph/ceph-client.admin.59684.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42345.log.gz 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59684.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42347.log 2024-05-09T08:59:11.094 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57055.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60577.log: /var/log/ceph/ceph-client.admin.42347.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60577.log.gz 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42347.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45699.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57055.log: /var/log/ceph/ceph-client.admin.45699.log: 88.3% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45699.log.gz 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57055.log.gz 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59348.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59348.log.gz 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54780.log 2024-05-09T08:59:11.095 INFO:teuthology.orchestra.run.smithi049.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44740.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58474.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54780.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47543.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42694.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58474.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58348.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56823.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58348.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47543.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58348.log.gz 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58810.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46602.log 2024-05-09T08:59:11.096 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56823.log: /var/log/ceph/ceph-client.admin.58810.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56823.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58810.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54822.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58558.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54822.log: /var/log/ceph/ceph-client.admin.46602.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46602.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54822.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60745.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58558.log: /var/log/ceph/ceph-client.admin.43673.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58558.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60745.log: /var/log/ceph/ceph-client.admin.42338.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60745.log.gz 2024-05-09T08:59:11.097 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42011.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42011.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42011.log.gz 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45783.log: /var/log/ceph/ceph-client.admin.54959.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41400.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43036.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59686.log 2024-05-09T08:59:11.098 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41400.log: /var/log/ceph/ceph-client.admin.43036.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41400.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43036.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46279.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43082.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59686.log: /var/log/ceph/ceph-client.admin.46279.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59686.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46279.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44123.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.44123.log: /var/log/ceph/ceph-client.admin.43082.log: 88.8% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43082.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44123.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59026.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42003.log 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.45301.log: /var/log/ceph/ceph-client.admin.59026.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59026.log.gz 2024-05-09T08:59:11.099 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35664.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41703.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42003.log: /var/log/ceph/ceph-client.admin.35664.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42003.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35664.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59902.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41526.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41703.log: /var/log/ceph/ceph-client.admin.59902.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41703.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58222.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59902.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46378.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58222.log: /var/log/ceph/ceph-client.admin.41526.log: 89.4% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41526.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58222.log.gz 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48976.log 2024-05-09T08:59:11.100 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-05-09T08:59:11.101 INFO:teuthology.orchestra.run.smithi183.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.46378.log: /var/log/ceph/ceph-client.admin.48976.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.46378.log.gz 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48976.log.gzgzip 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43505.log 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61308.log 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43505.log: /var/log/ceph/ceph-client.admin.42330.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43505.log.gz 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55183.log 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47851.log 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61308.log: /var/log/ceph/ceph-client.admin.55183.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61308.log.gz 2024-05-09T08:59:11.117 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55183.log.gz 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46896.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55061.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47851.log: /var/log/ceph/ceph-client.admin.46896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47851.log.gz 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46896.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43823.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53341.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55061.log: /var/log/ceph/ceph-client.admin.43823.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55061.log.gz 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43823.log.gzgzip -5 --verbose -- /var/log/ceph/objectstore_tool.51907.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61225.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53341.log: /var/log/ceph/objectstore_tool.51907.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.53341.log.gz 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41745.log 2024-05-09T08:59:11.118 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61225.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61225.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48535.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41745.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41745.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60367.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48535.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48535.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53502.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60367.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60367.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41128.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53502.log: 94.8% -- replaced with /var/log/ceph/objectstore_tool.51907.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41024.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53502.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59339.log 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41128.log.gz 2024-05-09T08:59:11.119 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.41024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41024.log.gz 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr: 88.0%/var/log/ceph/ceph-client.admin.43889.log: -- replaced with /var/log/ceph/ceph-client.admin.59339.log.gz 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54247.log 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.42009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gz 2024-05-09T08:59:11.120 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54247.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54247.log.gz 2024-05-09T08:59:11.146 INFO:teuthology.orchestra.run.smithi183.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T08:59:11.155 INFO:teuthology.orchestra.run.smithi049.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T08:59:11.161 INFO:teuthology.orchestra.run.smithi049.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T08:59:11.189 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T08:59:11.199 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T08:59:11.209 INFO:teuthology.orchestra.run.smithi049.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T08:59:11.218 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T08:59:11.256 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T08:59:11.258 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-05-09T08:59:11.258 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m0.308s 2024-05-09T08:59:11.258 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m1.399s 2024-05-09T08:59:11.258 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.068s 2024-05-09T08:59:11.259 INFO:teuthology.orchestra.run.smithi049.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T08:59:11.262 INFO:teuthology.orchestra.run.smithi049.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T08:59:11.282 INFO:teuthology.orchestra.run.smithi049.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T08:59:11.284 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-05-09T08:59:11.285 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m0.354s 2024-05-09T08:59:11.285 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m1.558s 2024-05-09T08:59:11.285 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.240s 2024-05-09T08:59:11.285 INFO:tasks.ceph:Archiving logs... 2024-05-09T08:59:11.285 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi049/log 2024-05-09T08:59:11.286 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T08:59:11.732 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi183/log 2024-05-09T08:59:11.733 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T08:59:11.931 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T08:59:11.948 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T08:59:11.948 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:59:11.950 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:59:12.034 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T08:59:12.043 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T08:59:12.043 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:59:12.045 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:59:12.059 INFO:teuthology.orchestra.run.smithi049.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:59:12.060 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +427us[ +428us] +/- 48ms 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -827us[ -826us] +/- 72ms 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +210us[ +210us] +/- 77ms 2024-05-09T08:59:12.064 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:59:12.065 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:59:12.065 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-05-09T08:59:12.066 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +429us[ +446us] +/- 49ms 2024-05-09T08:59:12.066 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -834us[ -834us] +/- 73ms 2024-05-09T08:59:12.066 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +224us[ +241us] +/- 77ms 2024-05-09T08:59:12.066 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:59:12.066 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T08:59:12.075 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T08:59:12.076 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T08:59:12.084 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:59:12.182 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:59:12.246 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:59:12.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:59:12.276 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:59:12.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:59:12.303 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T08:59:12.313 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T08:59:12.322 INFO:teuthology.task.internal:Duration was 716.832898 seconds 2024-05-09T08:59:12.322 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T08:59:12.331 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T08:59:12.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:59:12.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:59:12.372 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:59:12.379 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:59:12.716 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T08:59:12.716 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-05-09T08:59:12.716 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:59:12.778 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-05-09T08:59:12.778 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:59:12.807 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T08:59:12.807 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:59:12.821 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:59:12.872 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T08:59:12.872 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:59:12.888 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:59:13.016 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T08:59:13.028 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T08:59:13.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:59:13.054 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:59:13.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T08:59:13.087 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:59:13.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:59:13.118 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-05-09T08:59:13.138 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-05-09T08:59:13.165 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:59:13.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:59:13.187 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:59:13.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:59:13.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T08:59:13.229 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T08:59:13.230 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi049 2024-05-09T08:59:13.230 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:59:13.265 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698706/remote/smithi183 2024-05-09T08:59:13.265 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:59:13.295 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T08:59:13.296 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:59:13.302 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:59:13.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T08:59:13.357 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T08:59:13.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T08:59:13.367 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T08:59:13.367 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:59:13.369 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:59:13.383 INFO:teuthology.orchestra.run.smithi049.stdout: 265815 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:59 /home/ubuntu/cephtest 2024-05-09T08:59:13.400 INFO:teuthology.orchestra.run.smithi183.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:59 /home/ubuntu/cephtest 2024-05-09T08:59:13.401 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T08:59:13.412 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T08:59:13.438 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} duration: 716.8328981399536 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T08:59:13.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:59:13.524 INFO:teuthology.run:pass